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1.1.  Why  Slow? 


CHAPTER  1 
INTRODUCTION 


The  first  question  which  is  asked  when  the  topic  of  slow  adaptation  is  introduced  is  "Why 
slow?"  Here  are  several  answers  to  this  question. 


(1)  In  many  control  systems  the  plant  parameters  change  very  slowly  with  respect  to  the  time 
constants  of  the  closed-loop  system  with  fixed  controller  and  plant  parameters,  or  the  plant 
parameters  make  infrequent  step  changes.  For  such  systems  a  fixed  parameter  controller 
provides  good  performance  for  some  initial  interval,  but  performance  and  even  stability  can 
be  lost  as  the  plant  parameters  drift  from  the  initial  values.  This  situation  is  ideal  for  slow 
adaptation  which  can  either  continuously  retune  the  controller  parameters,  or  be  turned  on 
for  finite  intervals  as  an  on-line,  on-demand  tuning  algorithm. 


(2)  Using  slow  adaptation,  adaptive  control  systems  can  be  designed  for  given  controller 
parametrizations.  That  is.  the  controller  parametrization  is  chosen  as  necessary  for  the  design 
of  a  good  fixed  parameter  controller,  and  then,  a  parameter  update  law  is  designed  for  the 
given  parametrization.  This  contrasts  with  the  theory  for  fast  adaptation  which  requires  the 
use  of  a  particular  controller  parametrization  (typically  ARMA)  which  is  chosen  for  the  ease 
of  theoretical  parameter  convergence  analysis. 


(3)  Slow  adaptation  ;eplaces  the  exact  matching  conditions  found  in  the  theory  for  fast 
adaptation  with  a  compatibility  requirement  that  the  fixed  parameter  controller  can  be  tuned 
to  give  the  desired  performance  by  adjusting  only  the  parameters.  A  compatible  controller 
never  requires  more  parameters  than  the  exact  matching  conditions  and  usually  requires 
many  fewer  parameters.  The  reduced  number  of  parameters  reduces  the  number  of 
frequencies  which  inputs  to  the  system  must  contain  in  order  to  be  sufficiently  rich. 


1.2.  Summitry  of  Results 


The  systems  we  study  in  this  monograph  reduce  when  the  parameters  are  constant  to  linear, 
time-invariant  systems  driven  by  inputs  which  are  independent  of  the  parameters.  Letting  x 
denote  the  states  and  0  denote  the  parameters,  we  study  systems  in  the  form 


x  =  A(0)x  +  B(0)  w(t)  (1-1) 

9  =  ef(t.0.x)  (1.2) 

for  continuous  time  or.  for  discrete  time,  in  the  form 

x(k+l)  =  A(0(k))  x(k)  +  B(0(k))  w(k)  (1.3) 

0(k+l)  =  0(k)  +  e  f(k,0(k).x(k))  ,  (1.4) 


where  x  contains  the  states  of  the  plant,  the  model,  the  dynamic  controller,  and  any  filters  which 
process  signals  before  they  enter  the  adaptation  scheme,  and  where  w(t)  is  a  vector  input 
containing  the  reference  input  and  any  disturbances  entering  the  system. 

Slow  adaptation  is  forced  upon  the  system  (1.1 )-( 1.2)  by  choosing  e  small.  An  intuitively 
appealing  approximation  of  the  solutions  of  ( 1.1 )—( 1.2)  is  obtained  by  a  two-step  procedure.  First 
assume  that  0  is  constant  in  (l.l)  and  evaluate  the  solution  as  a  function  of  t  and  0.  Assuming  that 
A(0)  is  Hurwitz.  we  define  the  frozen  parameter  response  v(t ,0)  by 

t 

i/(t.0)=  /  e^X1"5’  B(0)  w(s)  ds  .  (1.5) 

— oo 

which  is  simply  the  response  of  the  linear  time-invariant  system  (1.1)  to  the  input  w(t)  with 
initial  condition  zero  at  time  t„  =  —  oo.  The  second  step  is  to  substitute  v(t.0)  for  x  in  (1.2),  that  is, 

0  =  €f(t.0.i/(t.0))  ■  (1.6) 

Then,  letting  0(t;t(,.0,)  denote  the  solution  of  (1.6)  with  initial  data  0(t„)  =  0o,  and  letting 
x(t:t„.0n.x„)  .  0(t:tn.0„.x,,)  denote  the  solution  of  (1.1  Ml. 2)  with  initial  data  x(t„)  =  x„  .  0(t„)  =  0„. 


the  approximation  is  given  by 
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0(t;to.0o,xo)  =  0(t:to.0o)  (1.7) 

x(t;to.0o.xo)  =  v(t.0(t:to.0o)) ,  (18) 

with  the  second  approximation  holding  only  after  an  initial  transient  in  x.  This  implies  that  the 
long-term  behavior  of  the  slowly  adapting  system  (1.1 )-( 1.2)  is  approximated  by  the  reduced- 


order  system  (1.6).  This  reduced-order  system  is  interpreted  as  the  parameter  update  law  without 
transients  in  x.  Because  the  frozen  parameter  response  v(t.0)  is  simply  the  steady-state  response  of 
(1.1)  with  0  constant,  it  is  easily  evaluated  and  understood,  hence.  (1.6)  is  useful  for  both  the 
analysis  and  design  of  slowly  adapting  systems.  This  approach  has  been  used  by  Astrom 
(1983.1984)  in  an  analysis  of  a  specific  adaptive  scheme. 

The  idea  of  using  the  reduced-order  system  (1.6).  which  ignores  the  initial  condition  on  x,  as  a 
model  for  the  complete  system  (1.1 )— ( 1.2)  is  similar  to  ignoring  the  boundary  layer  in  singular 
perturbations.  Kokotovic,  Khalil,  and  O'Reilly.  (1986).  However,  the  presence  of  time-varying 
input  w(t)  in  (1.1)  prevents  the  application  of  the  usual  singular  perturbation  techniques  for 
establishing  the  approximation  (1.7 )-( 1 ,8 ).  In  Chapter  2  and  in  Riedle  and  Kokotovic  (1986a).  we 
apply  integral  manifold  theory  to  the  study  of  (1.1 )-( 1.2)  and  prove  that  there  exists  a  function 
g(t.0:e)  with  the  property  that  if  x0  =  g(to.0o;e)  then  x(t:to.0o.xo)  =  g(t.0(t;to,0o,xo)  ;e)  for  t  ^  t0. 
We  also  show  that  along  solutions  of  (1.1 )-( 1.2)  with  x0  ^  g(to,0I>;€)  the  difference 
x(t)  -  g(t.0(t):e)  decays  exponentially  to  zero.  That  is.  for  certain  initial  conditions  or  after  the 
state  transient  decays,  the  reduced-order  system 

0  =  ef(t.9.g(t.0;e))  (19) 

and  the  algebraic  equation  x(t)  =  g(t.0(t);e)  provide  an  exact  description  of  the  slow  adaptation  of 
U  1  )-( 1.2)  This  f unction  g  defines  an  integral  manifold  M,  of  (1.1  Ml. 2)  by 

Vle  =  lt.0.x  :  x  =  g(t.0:e)|  (1  10) 

furthermore,  the  difference  h(t.0;e)  =  g(t.0:€)  —  i/(t.0)  between  the  function  g  and  the  frozen 
parameter  response  v  is  O(e).  Hence,  the  existence  of  \1,  implies  that  the  approximation  ( 1  7)-(  1 .8 ) 
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and  the  approximate  reduced-order  system  (1.6)  are  justified. 

Although  the  reduced-order  system  (1.9)  is  easier  to  study  than  the  complete  system  (1.1)- 
(1.2).  it  is  still  a  system  of  nonlinear  time-varying  differential  equations.  Noting  that  (1.9)  is  in  a 
standard  Bogoliubov  form  for  the  method  of  averaging,  the  solutions  of  (1.9)  (or  (1.6))  are 
approximated  by  the  solutions  of  the  time-invariant  nonlinear  differential  equation 

=7(S)  .  (1.11) 

dr 

where  t  =  €t  is  slow  time  and  where  f  is  the  time  average  of  f  for  fixed  9. 

i  ,+T 

f(fl)  =  lim  —  f  t(s.9.v(s.9))ds  .  (1.12) 

T— *co  1  * 

Hale  (1980).  Vleerkov  (1973),  Sethna  and  Moran  (1968),  Volosov  (1962).  and  Bogoliubov  and 
Mitropolski  (1961).  The  method  of  averaging  was  used  to  simplify  the  analysis  of  (1.6)  in  Astrom 
(1983.1984). 

The  method  of  averaging  gives  more  than  a  simplification  of  the  analysis.  By  interpreting  the 
stability  and  instability  conditions  provided  by  averaging  theory  in  terms  of  the  signals  and 
transfer  functions  in  the  adaptive  system,  we  developed  a  signal  dependent  stability  criterion  for 
slow  adaptation  of  the  Narendra  and  Valavani  (1978)  adaptive  control  algorithm  designed  for  a 
relative  degree  one.  order  n  plant  but  applied  to  a  plant  of  order  np  >  n  with  unspecified  relative 
degree.  Riedle  and  Kokotovic  (1985)  and  Kokotovic.  Riedle.  and  Praly  (1985).  At  that  time,  the 
integral  manifold  theory  had  not  yet  been  applied  to  (1.1 )-( 1.2);  hence,  the  transformation  of 
( 1.1 )-( 1.2)  into  the  standard  form  (1.9)  was  not  available.  The  stability  criterion  was  established 
by  linearizing  the  adaptive  system  (1.1 )-( 1.2)  and  then  performing  a  time-varying  transformation 
of  the  linearized  equations  into  a  standard  form  for  the  method  of  averaging.  This  transformation 
is  used  in  several  subsequent  works  which  also  obtain  local  results  via  averaging  theory.  Fu, 
Bodson.  and  Sastry  (1985).  kosut.  Anderson,  and  Mareels  (1985).  Anderson  et  al.  (1986).  and 
Bodson  et  al.  ( 1985).  After  showing  that  the  Narendra  and  Valavani  (1978)  controller  possesses  an 
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integral  manifold  under  slow  adaptation.  Chapter  2  concludes  with  a  more  complete  discussion  of 
the  stability  criterion  results  and  an  estimate  of  the  region  of  attraction  which  is  not  dependent  on 
linearization. 

In  view  of  the  similarity  between  the  continuous-time  system  (1.1 )-( 1.2)  and  the  discrete¬ 
time  system  (1.3)-(1.4).  it  is  tempting  to  simply  state  that  the  discrete-time  counterparts  of  the 
results  of  Chapter  2  hold  with  appropriate  modifications  of  the  proofs.  However,  this  claim  has 
met  with  some  skepticism  and  the  supporting  literature  for  ordinary  difference  equations  is  not  as 
extensive  as  that  for  ordinary  differential  equations.  Therefore,  we  take  this  opportunity  to 
present  in  Chapter  3  a  complete  self-contained  proof  of  these  results  for  the  discrete-time  slowly 
adapting  system  ( 1 .3 )-( 1 .4).  Our  proof  of  the  existence  of  an  integral  manifold  follows  the  proof 
in  Chapter  2  for  continuous-time  except  that  references  to  Chapter  VII  of  Hale  (1980)  are  replaced 
with  a  complete  derivation  of  the  required  bounds.  Using  a  different  proof.  Praly  (1986)  has  also 
shown  the  existence  of  an  integral  manifold  of  (1.3 )-( 1 .4). 

With  the  existence  of  an  integral  manifold  Me  established,  it  follows  that  the  system  (1.3)- 
( 1.4)  restricted  to  the  manifold  is  described  by  x(k)  =  g(k,0(k);e)  and 

0(k+l)  =  0(k)  +  ef(k.0(k).g(k.0(k):€)) .  (1.13) 

which  is  analogous  to  (1.9).  While  many  results  are  available  in  the  cited  literature  for  averaging 
of  the  ordinary  differential  equation  (1.9).  very  few  results  are  available  for  the  ordinary 
difference  equation  (1.13)  with  deterministic  inputs.  The  notable  exception  to  this  rule  is  Meerkov 
(1973)  who  states  theorems  for  discrete-time  systems  (but  refers  to  the  continuous-time  proof). 
Taking  inspiration  from  Meerkov’s  continuous-time  proofs,  we  state  and  prove  several  basic 
averaging  theorems  relating  the  solution  of  (1.13)  to  the  solutions  of  the  ODE  (l .1 1 )  with 

_  a  i  k+N— 1 

f(0)  -  lim  —  £  f(i,0.v(i.0))  .  (1.14) 

n-~  N 

where  v  is  now  the  frozen  parameter  response  of  (1.3).  In  contrast  to  the  averaging  theory  for 
(1  13)  with  deterministic  inputs,  many  articles  have  been  written  concerning  the  relationship 


between  the  behavior  of  (1.13)  and  the  ODE  (1.11)  when  (1.13)  represents  a  recursive  adaptive 
algorithm  with  stochastic  inputs.  Ljung  (1977),  Kushner  (1977).  and  Benveniste.  Goursat.  and 
Ruget  (1980),  to  mention  a  few.  Our  proof  is  easily  applied  to  a  stochastic  system  on  a  sample 
path  by  sample  path  basis.  Hence,  some  "with  probability  one"  results  for  the  system  (1.3)— (1.4) 
can  be  obtained  as  corollaries  of  our  basic  ayeraging  theorems.  However,  we  use  some  boundedness 
assumptions  and  many  triangle  inequalities  in  our  proof.  Hence,  our  proof  does  not  reproduce  any 
of  the  weak  convergence  results. 

Motivated  by  the  method  of  sensitivity  points  (Kokotovic.  1973)  and  some  early  work  on 
self-adaptive  systems.  Medanic  and  Kokotovic  (1965)  and  Kokotovic,  Medanic.  Vuskovic.  and 
Bingulac  (1966),  we  present  in  Chapter  4  and  in  Riedle  and  Kokotovic  (1986c)  a  controller 
parametrization  with  much  more  flexibility  than  the  parametrizations  usually  seen  in  the  adaptive 
control  literature  and  a  parameter  update  law  which  is  designed  with  the  intention  of  using  slow 
adaptation.  This  parametrization  allows  the  number  of  adjustable  gains  to  be  chosen  independently 
from  the  order  of  the  fixed  gain  feedback  controller.  Hence,  it  provides  the  freedom  to  design 
adaptive  control  systems  with  only  a  few  adjustable  parameters.  Along  with  this  freedom  comes 
the  necessity  (and  hence,  opportunity)  to  use  much  a  priori  information  about  the  plant  and  to 
make  a  more  extensive  off-line  design  effort.  The  additional  design  effort  is  compensated  by 
improved  performance  and  confidence  in  the  on-line  operation  of  the  slowly  adapting  system. 

As  noted  earlier,  slow  adaptation  allows  the  controller  parametrization  to  be  specified  for  the 
design  of  a  good  fixed  parameter  controller.  After  the  controller  parametrization  is  specified,  the 
design  of  the  slowly  adapting  system  is  completed  by  developing  a  parameter  update  law.  In 
Chapter  5  we  illustrate  the  development  of  a  parameter  update  law  for  a  given  controller 
parametrization.  The  plant  is  fifth  order  with  three  uncertain  parameters  and  the  controller  is  first 
order  with  three  adjustable  parameters.  The  uncertain  plant  parameters  can  vary  from  given 
nominal  values  by  30%..  We  first  do  off-line  numerical  analysis  to  verify  that  the  controller  can  be 
tuned  for  all  possible  values  of  the  plant  parameters.  We  then  present  simulation  results  which 
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show  that  the  controller  parameters  of  the  slowly  adapting  system  converge  to  the  values  which 
achieve  optimal  tuning  in  the  off-line  numerical  analysis.  We  conclude  by  showing  that  the 
theoretical  analysis  of  this  algorithm  is  similar  to  the  analysis  in  Chapter  4  and  that  the  positive 
simulation  results  are  predicted  by  the  analysis. 


CHAPTER  2 


INTEGRAL  MANIFOLDS  OF  SLOW  ADAPTATION  IN  CONTINUOUS  TIME 

2.1.  Introduction 

Continuous-time  adaptive  algorithms  for  estimation  and  control  can  be  represented  by  the 
nonlinear  dynamic  system 

x  =  A(0)x  +  B(0)w(t)  x€Rn*,w€Rr  (2.1) 

0  =  ef(t.fl.x).  0  €  R>  (2.2) 

The  x-equation  (2.1).  where  w(t)  incorporates  both  the  reference  and  disturbance  inputs,  describes 
the  plant,  its  controller,  filters,  etc.;  hence,  the  nx-vector  x  is  referred  to  as  a  "state."  The  In¬ 
equation  (2.2)  is  the  update  law  for  the  ng-vector  of  adjustable  "parameters."  When  x  and  0 
strongly,  interact  the  distinction  between  the  "states"  and  "parameters"  is  meaningless.  However,  in 
the  case  of  "slow  adaptation"  this  distinction  is  meaningful  and  greatly  simplifies  the  analysis.  In 
the  system  (2.1 )-( 2.2 )  the  slow  adaptation  is  due  to  the  smallness  of  the  scalar  gain  e,  which  forces 
0  to  be  small  and  the  parameters  0  to  evolve  slowly  compared  to  the  states  x.  Even  without  this 
scaling  by  €.  a  typical  adaptive  transient  consists  of  a  few  rapid  initial  swings  after  which  the 
parameters  continue  to  move  slowly  as  f(t,0.x)  becomes  small.  During  the  period  of  slow 
adaptation  the  parameters  may  (a)  remain  in  a  bounded  set  where  Re  A(A(0))  <  0.  (b)  drift  toward 
infinity  with  Re  A(A(0))  <  0.  or  (c)  drift  to  a  region  where  Re  A(A(0))  >  0. 

In  this  chapter  the  concept  of  slow  adaptation  is  made  precise  by  showing  that  it  occurs  in  an 
integral  manifold  Yle  of  (2.1)-(2.2).  a  time-varying  n#-dimensional  surface  in  the  nx  +  n<r 
dimensional  space  of  x  and  0.  defined  by 

Mf  =  !t,0.x  :  x  =  g(t,0:e))  .  (2.3) 

where  i/(t.0)  =  g(i.0:(»  is  the  steady-slate  response  of  (2.1)  with  constant  0.  In  Section  2.2  we 
show  that  g(t.0:e)  can  be  viewed  as  a  similar  steady-state  response  in  the  case  of  slow  variations  of 
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9.  For  this  reason  we  call  M(  a  "slow  manifold"  of  (2.l)-(2.2).  The  motion  of  the  parameters  9  in 
the  slow  manifold  is  governed  by  the  update  law  (2.2).  but  with  x  replaced  by  g(t.0;e).  that  is. 

9  =  efCt.fl.  g(t.0:6»  .  (2.4) 

For  an  adaptive  system  this  equation  is  an  exact  description  of  the  adaptation  process  after  the  state 
transients  have  decayed. 

In  Section  2.3  we  formulate  conditions  for  the  existence  of  h(t.0:«)  =  g(t,0;e)  —  i/(t.0).  and  in 
Section  2.4  we  give  conditions  for  the  slow  manifold  M*  to  be  attractive,  as  well  as  a  procedure  for 
estimating  the  region  of  attraction.  By  showing  that  M,  is  attractive  and  that  h(t.0:e)  -♦  0  as  €  -*  0, 
we  justify  the  use  of 

9  =  6f(l.«.v(t.0))  (2.5) 

as  an  approximation  of  (2.4)  for  €  sufficiently  small.  This  approximation  combined  with  an 
averaging  analysis  of  (2.10)  was  instrumental  in  Astrom's  (1983.1984)  lucid  explanation  of  the 
drift  instability  observed  by  Rohrs  et  al-  (1982.  1985).  In  Section  2.5  and  in  Riedle  and  Kokotovic 
(1986b)  we  show  that  the  given  conditions  for  the  existence  of  an  attractive  slow  manifold  are  met 

by  a  standard  model  reference  adaptive  control  system.  The  results  of  Section  2.6  prove  the 

asymptotic  validity  of  Astrom's  approach  and  generalize  the  analysis  which  led  to  the  local 
stability  criteria  formulated  by  Riedle  and  Kokotovic  (1985).  Kokotovic.  Riedle.  and  Praly  (1985). 
Kosut.  Anderson,  and  Mareels  (1985).  Fu.  Bodson,  and  Sastrv  (1985),  and  Riedle.  Praly.  and 
Kokotovic  ( 1986). 

Before  we  proceed,  let  us  mention  that  the  concept  of  an  integral  manifold  was  introduced  by 
Lyapunov  and  used  by  him  and  Perron  in  their  studies  of  conditionally  stable  systems.  More 
recently  this  concept  is  encountered  in  the  averaging  literature,  Bogoliubov  and  Mitropolski  ( 1961 ). 
Volosov  ( 1962).  and  Mitropolski  and  Lykova  (1973).  A  comprehensive  treatment,  independent  of 
averaging,  is  found  in  Pliss  (1966.  1977)  and  Hale  (1980).  Closely  related  notions  are  center 


manifolds:  Fenichel  (1971)  and  Carr  (1981),  and  singular  perturbations:  Hoppensteadt  (1971). 
Fenichel  (1979),  and  Kokotovic.  Khalil,  and  O'Reilly  (1986). 

2.2.  Interpretation  and  Approximation  of  the  Slow  Manifold 

An  integral  manifold  of  (2.1)-(2.2)  is  simply  defined  by  the  statement  that  if  the  vector 
x.9  is  in  at  t  =  t0.  then  it  is  in  Mt  for  all  t.  that  is. 

|£o|6M<  Vl  (2'6) 

If  a  manifold  Me  can  be  found  for  each  value  of  6  in  a  segment  e  €  [O.c’].  then  we  shall  say  that  an 
e-family  of  slow  manifolds  exists.  The  simplest  member  of  this  family  is  the  "frozen  parameter" 
manifold  M„  defined  by  €  =  0  and  the  requirement  that  if  x  =  v(t.O)  at  r  =  t.  then  x  =  v(t.  ))  for  all 
r  €  R.  \oting  from  (2.2)  that  if  €  =  0.  then  9  is  constant,  we  have 

t 

v(l.9)  =  eA(9Xl-T>y(s,0)  +  f  eA<9Xl~T>B(0)w(T)dT.  (2.7) 

S 

Assumption  2.1  :  There  exist  a  set  ©  C  R"9  and  constants  a  >  0  and  K  ^  1  such  that 


|  eA(flXt-s>  |  ^  Ke-0' 


-Ofllt-s) 


where  |  ■  |  is  the  Euclidean  norm. 


Under  this  assumption  we  let  s  -*  —  oo  and  obtain  from  (2.7) 


i/(t.0)  =  /  eMe*'~r)  B(0)w(r)dT. 


For  a  broad  class  of  signals  w(t).  including  almost  periodic  signals,  this  integral  is  well  defined  and 
shows  that  \ln  represents  the  familiar  "steady-state  response"  of  the  linear  time-invariant  system 
(2.1  )  considered  as  a  function  of  both  t  and  9.  Although  the  situation  is  more  complicated  when 
€  >  0  and  9  is  not  constant,  the  interpretation  of  M  as  a  "steady-state  response"  is  still  helpful. 


A A  A  A  • A  >Vc 1 <  -A.A.A. '.A.V 


Introducing  the  deviation  of  x  from  i/(t,0)  as  a  new  state  variable 


z  =  x  —  v(t  .0). 

(2.10) 

we  rewrite  (2.1  )-(2.2)  in  the  form 

z  =  A(0)z  —  €Ve(t.0)F(t.0,z), 

(2.11) 

0  =  €f(t,0,i/(t.0)  +  z)  =  eF(t.0.z). 

(2.12) 

where  v#(t.0)  is  the  nxXn#  sensitivity  matrix 

Vfl(t .0)  =  |J-  =  /  €**'-*>  -jfr  [A (0)x  +  B(0)u(r)]x  =  J,<r.wdT.  (2.13) 

The  brackets  indicate  that  is  performed  with  x  fixed,  after  which  u(t.Q)  is  substituted  for  x. 

Properties  of  the  response  v(t.0)  and  its  sensitivity  VgCt.O)  are  among  the  crucial  factors  influencing 
the  behavior  of  an  adaptive  scheme.  We  characterize  these  properties  by  assuming  bounds  on  v  and 

v*. 

Assumption  2.2:  There  exist  positive  constants  v,  V],  and  v2  such  that  for  all  t  €  R  and  0.069 

|v(t,0)|<v.  |  Vfl(t.0)|  <  Vj.  |  */e(t.0)  —  Pw(t.0)l  ^  v2  |0  —  0|.  (2.14) 

□ 

Remark  2.1:  A  sufficient  condition  for  Assumption  2.2  to  be  satisfied  is  that  A(0).  and  B(0)  have 
Lipschitzian  derivatives  and  that  w(t)  is  uniformly  bounded.  We  make  the  assumption  directly  on 
v  to  Vq  to  simplify  expressions  in  this  sequel. 

□ 

In  the  (z,0)-coordinates  \l(l  is  defined  by  z=0.  To  define  M  for  €  >  0  we  need  to  find  a  function  h  of 
t  and  0  parametrically  dependent  on  €  such  that 

z  =  h(t.0;e)  (2.15) 

satisfies  (2.11  )-(2. 12).  Let  us  first  interpret  h(t,0;e)  by  constructing  a  sequence  of  "steady-state 
responses"  ho(t.0;€).  h,(t.0:e) . hk(t.0:€),...  which  in  Section  2.3  will  be  shown  to  converge  to 
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h(t.0;€).  Suppose  that  hk(t.0;€)  is  available  for  all  t  and  each  0  €  0.  and  substitute  it  for  z  in 
(2.12).  Then  compute  the  solution  0k(s;t.0,€)  of  the  end-value  problem 

—  ek(s)  =  €F(s.ek(s).hk(s.ek(s:«))).  ek(t)  =  0  €  e .  (2.16) 

ds 

With  0k(s)  =  0k(s:t.0.€)  and  hk(s,0k(s))  available,  use  (2.11)  to  evaluate  hk+1  along  0k(s)  from 

—  hk+1(s.0k(s):€)  =  A(0k(s))hk+1(s.0k(s):e)  -  €i/e(s.0k(s))F(s.0k(s).hk(s.0k(s):e)) .  (2.17) 


The  state  transition  matrix  <l>k(s.T)  =  4>k(s.r;t,0.e)  of  the  linear  time-varying  system  (2.17)  is 
defined  by 

A.  4>k(s.r)  =  A(0k(s))4>k(s.T).  4>k(r.r)  =  1.  (2.18) 

ds 

If  (2.18)  is  exponentially  stable,  that  is.  if  as  in  (2.8).  there  exist  positive  constants  Kj  and  ai  such 
that 

|*k(s.r)|  Kle~",ls_T).'  ¥s  >  r.  Vr  €  R.  (2.19) 

then  the  steady-state  response  of  (2.17).  analogous  to  (2.9).  is 

S 

hk+1(s.0k(s))  =  -€  f  <l>k(s.r>fi(T.0k(T))F(T.0k(r).hk(T.0k(r)))dT.  (2.20) 

— oo 

This  expression  defines  hk+i  along  a  particular  trajectory  0k(s)  whose  "end-point  at  s=t  is  0.  By 
choosing  different  "endH-points  0  €  0.  hence,  different  trajectories  0k(s)  =  0k(s:t.0.e).  the  function 

t 

hk+1(t.0;€)  =  -€  f  <!>k(t.r)i'e(T,0k(r))F(T,0k(r).hk(r.0k(T)))dr  (2.21) 

— oo 

can  be  evaluated  for  each  0  E  0  and  all  t  €  R.  Except  lor  the  use  of  different  trajectories 
0k(r )  =  0k(r;t.0,e)  in  place  of  different  constant  values  of  0.  there  is  a  clear  analogy  between 
hk+|(t.0:€)  defined  by  (2.21)  and  v(t,0)  defined  by  (2.9).  Initialized  with  ho(t.0:e)  =  0.  the 
sequence  hk(t.0:e).  k=l,2....  is  uniquely  defined  by  (2.16)  and  (2.21).  These  expressions,  which  are 
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If  a  continuously  differentiable  h(t,0;«)  is  known  to  exist,  then  the  substitution  of 


•  =  3h  +  g 

at  W 


(2.22) 


into  (2-11 )-( 2.12)  shows  that  h(t.0:€)  satisfies  the  partial  differential  equation 


4*  =  A(0)h  -  e(V6(i.6)  +  i£.)F(t.e.h) 

at  afl 


(2.23) 


which  suggests  that  it  may  be  computationally  feasible  to  approximate  h(t.0;«)  by  a  power  series 
in  €.  Substituting 


h(t.0:e)  =  h°(t.0)  +  ehKt.0)  +  e2h2(t.0)  + 


(2.24) 


into  (2.23)  and  evaluating  the  terms  of  the  series,  we  see  that  h°(t.0)  =  0  and  that  hKt.O)  is  the 
steady-state  response  of 


=  A(0)h1-i/e(t.0)F(t.0.O). 


(2.25) 


The  equations  for  h2.  h3.  etc.  are  more  complicated  and.  from  a  practical  point  of  view,  the 
approximation  h(t.0:e)  =  €h3(t.0)  may  be  all  that  is  needed  to  improve  the  "frozen  parameter" 
approximation  (2.5).  because  h3(t.0)  incorporates  the  effects  of  i/#(t.0),  which  are  important  when 
the  sensitivity  of  the  plant  with  respect  to  adjustable  parameters  is  high. 

Example  2.1:  The  analysis  of  the  effects  of  an  unstable  zero  —  >  0  on  the  performance  of  an 

/* 

adaptive  controller  designed  for  a  minimum  phase  plant  is  nontrivial  even  in  the  case  of  a  first 
order  plant  and  a  single  adjustable  parameter.  Such  an  adaptive  system,  shown  in  Fig.  2.1,  is 
described  bv 


1-0  .  1  ~fX 

X  =  -R^X+  w^r 


(2.26) 


a  -  c  x~Mr  (  x~Mr  __  w  ^ 
0-6 yJ 


(2.27) 


where  r  =  r(t)  and  ym  =  vm(t)  are,  respectively,  the  reference  input  and  model  output.  For 
r  =  costat  the  frozen  parameter  response  v(l.d)  and  its  sensitivity  are 


Fig.  2.1.  Block  diagram  of  the  system  (2.26)-(2.27). 

v(i.9)  =  V(0,o))cos(ti>t  +  0(0.  o>)) 


V(0.w)  =  __ 


1  -M 


7(1  —  /jl9)2  u2  +  (0—  l)2 


,  0(0.ti>)  =  arctan 


(1  —  fi9)ot 

0-  1 


Vy(i.9)  =  cos(ou  +  0)  —  V  sin(tot  +  0). 

With  j/  and  known,  h1  can  be  obtained  as  the  steady-state  response  of  (2.25). 
v(t.0)  +  €h'(t.0)  can  serve  as  an  approximation  of  h(t,0;e).  which,  in  this  case,  is  periodic  in  t 

For  a  clear  graphic  display  let  us  consider  the  constant  input  case  with  r  =  1.  ym  = 
simple  calculation  gives 


and.  upon  the  substitution  into  (2.24),  we  obtain 


h1  = 


l 


2(0  -  l)5 


(1  -m)(1  -M0X3-0). 


(2.28) 

(2.29) 

(2.30) 

Then 

.5.  A 

(2.31) 

(2.32) 


For  0  ^  2.25  the  sensitivity  Vq  is  small  and.  as  predicted,  the  slow  manifold  is  practically 
indistinguishable  from  v(9).  For  0  ^  2.25  the  trajectories  plotted  for  /jl  =  .25  and  €  =  1  in  Fig.  2.2 


clearly  show  a  separation  of  time  scales:  the  slope  -jg-  is  much  steeper  away  from  the  manifold 

than  along  the  manifold.  Again  as  predicted,  the  situation  changes  in  the  region  0^2  where  the 
sensitivity  v$  is  high.  In  this  region,  the  curve  v(9)  +  eh1(9)  is  a  significantly  better  approximation 

•J 

of  the  slow  manifold  than  v{9).  The  disastrous  effect  of  the  unstable  zero  —  is  also  characteristic: 

M 

J  1  1 

for  9  >  —  the  manifold  is  repulsive,  whereas  for  9  <  — .  it  is  attractive.  For  0  =  —  the  system 

M  n  n 

(2.26)-(2.27)  is  not  defined.  In  the  manifold,  the  slow  adaptation  converges  to  the  equilibrium 

0  =  3. 

Remark  2.2:  To  avoid  excessive  numerical  sensitivity  of  the  unstable  trajectories  for  0  >  4  .  they 
have  been  obtained  bv  simulation  in  reverse  time. 


Manifold 


Fig.  2.2.  Trajectories  of  (2.26)-(2.27 )  with  r  =  1.  yin  =  0.5.  6  =  0.1.  /j.  =  0.25  converge  to  the 
mamtold  for  0  <  4.  In  the  manifold  all  movement  is  toward  0  =  3. 


Remark  2.3:  An  analytical  study  of  repulsive  manifolds  would  involve  the  following 

modifications.  If  Re\i(A(0))  >  0  for  all  i=  1 m  .  then  (2.1l).(2.13).  and  (2.20)  are  to  be 

integrated  in  reverse  time,  from  co  to  t.  If  A  has  both  stable  and  unstable  eigenvalues,  then  each  of 

these  expressions  would  include  two  integrals,  one  from  —  oo  to  t  for  the  stable  part  and  one  from 

oo  to  t  for  the  unstable  part  of  the  response.  We  restrict  our  analysis  to  attractive  slow  manifolds. 

□ 

2.3.  Existence  of  the  Slow  Manifold 

Expressions  (2.16)  and  (2.21)  rewritten  as  hk+1  =  Thk  define  a  map  T.  Its  fixed  point,  if  it 
exists,  is  our  function  h(t.0;e).  As  in  any  fixed  point  argument,  we  first  specify  a  closed  subset  of  a 
Banach  space  in  which  to  search  for  h(t.0;e).  We  let  this  space  be  the  set  of  continuous  functions 

H(t.0)  equipped  with  the  norm  II  H  II  =  sup  |  H(t.0)|  and  use  positive  constants  D  and  A  to 

t  «€  r  x  r"9 

define  our  closed  subset  H(D.A)  as 

H(D.A)  =  (H:RxRn— Rm  I  IIHII^D  and  I  IKt.fl)  -  H(t.0) A|0  -  0|.  Vt6R:  0.0€R,le}.  (2.33) 

Our  goal  is  not  only  to  establish  the  existence  of  h(t.0.€).  but  also  to  characterize  it  by  estimating  D 
and  A  in  terms  of  €  and  the  data  about  the  adaptive  system.  In  addition  to  Assumptions  2.1  and  2.2 
about  the  system  (2.1)  with  constant  0,  we  need  the  following  data  about  the  parameter  update 
law  (2.12). 

Assumption  2.3  :  There  exist  nondecreasing  positive  functions  pF(D).  ps(D).  and  pz(D)  such  that 

I  F(t.0.z)  I  ^  pF(D).  I  F(t.0.z)  —  F(t,0,z)  |  ^  p*(D)|0  — 0|. 

(2.34) 

lF(t.0.z)  — F(t.0.z)|  pz(D)|z  — z|.  ¥t  €  R:  0.0  e  0:  |z|.|z|^D. 

□ 

Remark  2.4:  A  sufficient  condition  for  Assumption  2.3  to  hold  is  that  f(t,0.x)  be  bounded,  and 
I  ipschitzian  in  0.x  uniformly  with  respect  to  i€R  .  0€0  in  x  in  compact  sets.  These  are  very  mild 
conditions  w  inch  are  met  by  most  parameter  update  law  s. 

□ 
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V  , 


In  the  iterative  scheme  (2.16),  (2.21)  the  stability  condition  (2.19)  is  crucial.  Unfortunately, 
our  Assumption  2.1  is  not  sufficient  to  guarantee  that  each  trajectory  (^(sh.O.c)  will  remain  in  the 
set  0  for  all  s,  as  required  by  (2.19).  Following  Carr  (1981).  we  avoid  this  difficulty  by  proving 
the  existence  of  h(t,0',€)  for  a  modified  system,  rather  than  for  (2.1 1  )-(2.12).  In  the  modified 
system,  each  0  which  is  not  in  0  is  replaced  by  some  value  p(0)  which  remains  in  0.  By 
construction,  the  original  and  the  modified  systems  are  identical  for  all  0  6  0.  that  is.  p(0)  =  0. 
¥0  €  0.  For  simplicity,  we  restrict  the  set  0  in  Assumption  2.1  to  be  convex  and  compact.  Then 
p(0)  is  uniquely  defined  as  the  point  in  0  closest  to  0,  namely. 


& 


s'. 


I 


m 


p(0)  =  arg  min  |  p  -  0  | .  (2.35) 

p€  © 

We  henceforth  analyze  the  modified  system 


z  =  A(p(0))  —  ei/a(t,p(0))  F(t.p(0).z). 

(2.36) 

0  =  €F(t.0.p(0).z). 

(2.37) 

The  results  obtained  for  the  modified  system  translate  into  those  for  the  original  system  as 
follows.  Suppose  that  (2.36)-(2.37)  has  an  integral  manifold  h(t.0:e)  and  a  solution  0(t)  of  (2.37) 
with  z  =  h(t.0;e)  which  satisfies  0(t)  6  0.  Vt  6  [t„.tj.  Then  0(t).  z  =  h(t.0(t):e)  is  also  a  solution 
of  the  original  system  (2.11 )-( 2.12)  Vt  6  [t0.t,].  As  for  the  modified  assumptions,  we  note  that, 
because  p(0)  6  0  and  I  p(0)  —  p(0)  |  ^  |0  —  0  I.  the  bounds  imposed  on  A(0)  ,  v{9).  v«(t .0).  F(t,0.z) 
for  all  0  6  0  are  satisfied  by  A(p(0)).  i/(t.p(0)),  i/4(t.p(0)),  F(t.p(0).z)  for  all  0  6  RUs.  To  describe 
the  dependence  of  F(t.p(0).z)  and  i'*(t.p(0))  F(t,p(0).z)  on  0  over  the  set  H(D.A)  we  define  pj(D.A) 
and  p2(  D.A  )  such  that 

1  F( t.p(0).H(t.0))  —  F( t.p(01.H(t.0))  I  ^  p,(D.A)  i  0  —  0  I  (2.38) 

!  i'.,(t.p(0))F(t.p(0).H(t.0))  —  i/,.(t.p(0))F(t.p(0).H(t.0))l  ^  p2( I). A )  1 0  —  0  i  (2.39) 

tor  all  t  6  R.  0.0  6  R‘v  and  H  6  H(D.A)  It  follows  from  Assumptions  2.2  and  2.3  that  p,  and  p2 
exist  and  can  be  chosen  to  satisfy 


Pl 


ft 


-/VV-.' 

<  V  J  n 


s. 


V  *  V*  V  '  ’  *-*  * 

lA  aTi  ■ 


Pi(D.A)  <  p0(D)  +  Apz(D).  p2(D.A)  ^  vlPl(D.A)  +  v2pF( D). 


(2.40) 


We  now  perform  the  same  modification  of  the  iterative  expressions  (2.16)  through  (2.21).  In 
particular.  (2.16)  becomes 

d0k(s) 

— j — =  eF(s.p(0k(s)).  hk(s.0k(s);€)),  0,(1)  =  0.  (2.41) 

ds 

As  before,  the  trajectory  0k(s:t.0.e)  is  determined  by  its  ’,endH-condition  0  at  s=t.  However,  the 
modification  now  guarantees  that  to  each  hk  €  H(D,A)  and  each  0  €  R"6  there  corresponds  a  unique 
continuous  solution  of  (2.41)  0k(s:t.0,€)  =  0k(s),  defined  for  all  s  €  R.  This  is  a  consequence  of  the 
global  character  of  (2.38).  A  more  important  advantage  of  the  modification  is  that  the  stability 
property  (2.19)  of  <l>lc(s.r;t.0.€),  the  solution 

<tk(s.r)  =  A(p(0k(s)))  <t>k(s.r).  4>(t.t)  =  I.  (2.42) 

can  be  established  as  follows. 

Lemma  2.1  :  Suppose  that  the  Assumptions  2.1,  2.2.  and  2.3  hold:  choose  a  constant  a  >  0  such 
that  j  A C 0 )  —  A(0)  |  ^  al  0  —  0  I  for  all  0.0  €  0  and  let 


p3(D)  =  [app(D)  K  Ink]*. 


(2.43) 


If  hk  €  H( D.A )  and 


€apF(D)  <  a2(k  Ink)  1 


(2.44) 


then 


I<Ms.t)|  ^  Ke-“.<t  D)(s-T)  .  Vs  ^  r 

c  here  Q|(€.D)  =  q„  —  £  p,(D). 


(2.45) 


Proof  :  By  assumption,  for  all  s.s  6  R. 

A(  pi  0k(  si) )  —  A(  p(0k(s)))  |  $  al  0k(s)  —  0k(s)  |  5;  eapF(  I) )  |  s  —  si.  (2.46) 

and  h\  construction,  A(p(0))  satisfies  (2.8)  for  all  0  €  R'H  The  proof  then  follows  from  a  standard 


.■  - 


theorem  for  systems  with  slowly  varying  coefficients,  e.g..  page  117  of  Coppel  (1965)  or  Section  2.5 
of  Kokotovic.  Khalil,  and  O'Reilly  (1986). 

□ 

We  are  now  prepared  to  consider  the  map  T  defined  pointwise  via 

t 

(Thk)(t,0:e)  =  -€  /  4>k(t.r)  i/e(r.p(0k(T)))  F(r.p(0k(r)).hk(r.0k(T):e))  dr  .  (2.47) 

-oo 

where  0k(r)  —  0k(r.t.0.€)  and  4>k(t,r)  =  4>k(t,T;t.0.e).  The  meaning  of  T  is  made  clear  by  comparing 
(2.47)  with  (2.21),  that  is.  the  map  T  represents  the  iterations  (2.16)  through  (2.21)  for  the 
modified  system  (2.36)-(2.37). 

Lemma  2.2  :  Suppose  that  Assumptions  2.1,  2.2,  and  2.3  hold.  If  €,  D.  and  A  satisfy  (2.44)  and 


eKvjPp(D)  /cKi(e.D)  ^  D 


ep,(D.A)  <  aj(eD) 


p2(D. 


KavlPF(D) 

A)  + - ^  A 

ot  jU.D) 


pz(D)  . 

6  [Kv'  +  41  <  1 


(2.48) 


(2.49) 


(2.50) 


(2.51) 


then  T  is  a  contraction  mapping  on  H(D.A). 

Proof( discussion):  Omitting  lengthy  calculations  of  the  bounds  (2.48)-(2.51 ),  we  only  indicate 
their  origin  and  discuss  their  meaning.  Using  (2.45)  and  (2.47)  it  is  not  difficult  to  see  that  (2.48) 
assures  ll  Thk  II  <  D  The  most  complicated  bound  (2.50)  originates  from 
(Thk)(t.0:e)  —  (Thk)( t.0;6)  written  as  the  sum  of  two  integrals 

l 

-6  /  <!>k(t.T)K(T.p(0k(T)))F(T.p(0k(T)).hk(T.0k(T))) 

— rxn 

-  ^(T.p(0k(r)))F(T.p(0k(r)).hk(r.0k(r)))]dT  (2.52) 

t 

-€  /  [cDk(t.r)-<i>k(t.r)K(r.p(0k(T)))F(r.p(0k(T)).hk(T.0k(T)))dT 


I'rtTrriTrtfiy** 


where  0k(r)  =  0k(T:t,0,e)  and  <J>k(t,r)  =  <l>k(t.T:t.0.e).  Now  (2.49)  assures  that  the  norm  of  the  first 
integral  is  bounded  by 

|e-e]Kp2(D.A)[a1(6.D)-€p1(D.A)]-1  .  (2.53) 

hence,  is  well  defined  over  any  infinite  interval  (— oo.t].  The  term  in  the  brackets,  also  appearing  in 
(2.49)  is  of  conceptual  interest,  because  it  represents  a  time-scale  separation  property.  To  see  this, 
note  from  (2.38)  and  (2.41)  that  I  0k(s)  —  0t(s)  I  ^  10  —  01  e**>i(D,A)ls  1  ;  hence.  epi(D.A)  is  the 
fastest  exponential  rate  of  the  "steady-state"  solutions.  On  the  other  hand.  (2.45)  shows  that 
a,(€.D)  is  the  slowest  exponential  decay  towards  a  "steady-state"  solution.  If  the  difference  of 
these  two  rates  is  larger,  the  dependence  of  h  on  0  will  be  "smoother."  The  other  term  in  (2.50) 
indicates  that  the  smallness  of  the  sensitivity  bound  V!  also  contributes  to  the  "smoothness"  of  h. 
Finally.  (2.51 )  is  a  "contraction"  bound  for  II  Thk  —  Thm  II  /  II  hk  —  hm  II .  For  further  details  in  this 
continuous-time  case  see  Chapter  VII  of  Hale  (1980).  We  give  a  complete  proof  following  Hale  for 
the  discrete-time  case  in  Chapter  3.  To  conclude,  let  us  mention  that  the  time-scale/smoothness 
relationship  is  clarified  in  Fenichel  (1971). 

□ 

Remark  2.5  :  The  only  use  of  (2.44)  in  Lemma  2.2  is  to  ensure  (2.45)  holds  with  aj(e.D)  >  0.  If 
(2.45)  can  be  established  for  a^e.D)  >  0  without  (2.44),  then  (2.44)  can  be  dropped  as  a 
hypothesis  of  Lemma  2.2.  For  example,  if  there  exists  a  constant  positive  definite  symmetric 
matrix  P  which  satisfies 

At(0)P  +  PA(0)  ^  -a„P  ¥0  €  0  .  (2.54) 

then  (2.45)  is  satisfied  with  aijfe.D)  =  c*u  and  K  =  (A.maxP/\mmP)v. 

□ 

It  is  clear  that  for  any  positive  Du  ,  ai(e,D„)  will  be  positive  for  €  sufficiently  small.  With 
this  observation  in  mind,  it  is  obvious  that  ( 2.45 )-( 2.51)  will  be  satisfied  lor  any  positive  D,,  .  A,, 
by  a  sufficiently  small  >  0,  hence,  for  all  6  6  [().€„].  In  view  of  the  fact  that  Pj.  and  p7  are 
nondecreasing  functions  of  D.  it  is  clear  that  for  e  <  e(,  we  can  use  D  =  (e/e,,)  D,,  and  A  =  (e/e,,)  A,, 
instead  of  D,,  and  A,,  in  the  definition  of  H.  Hence,  under  the  conditions  of  Lemma  2.2  the  function 
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h(t.0:e)  exists  and  is  an  0(«)  quantity.  This  observation  leads  to  the  following  summary  of  our 
existence  results. 

Theorem  2.1:  Suppose  that  Assumptions  2.1.  2.2.  and  2.3  hold.  Then,  given  any  fixed  D0  >  0  and 
A„  >  0.  there  exists  €,,  >  0  such  that  for  each  e  e  [0,€0]  the  modified  system  (2.36)-(2.37)  has  an 
integral  manifold  uniquely  defined  by 

\1*  =  [t,0,z  :  z  =  h(t,0:e)|,  h  €  H  ((€/€<>)  D0  .  (e/€0)  A0).  (2.55) 

□ 

When  translated  to  the  original  system,  this  result  establishes  the  existence  of  an  €-family  of 
slow  manifolds  of  (2.1M2.2).  Recalling  that  x  =  z  +  v(t,0).  that  g  =  v  +  h.  and  that  in  x.0 
coordinates  M(  =  [t.0,x  :  x  =  g(t.0:€)}  we  translate  Theorem  2.1  to  the  original  system  in  the 
following  corollary. 

Corollary  2.1  :  Suppose  that  Assumptions  2.1.  2.2.  and  2.3  hold.  Let  x(t),  0(t)  be  tne  solution  of 
(2.1 )-( 2.2 )  with  initial  data  0(t„)  =  0O  €  9  and  x(t„)  =  g(t„.0o;e)  =  */(t„.0o)  +  h(to.0„;e).  Suppose 
that  0(t)  €  9  for  all  t  €  [t0.t,].  Given  any  fixed  D,>  >  0  and  A0  >  0.  there  exists  60  >  0  such  that 
foreache  €  [O.eJ  the  solution  x(t).  0(t)  of  (2.1  )-(2. 2)  satisfies 

(t.0(t).x(t))  €  Me  =  {t.0.x  :  x  =  g(t.0:e)  =  Kt.0)  +  h(t.0:e)}  (2.56) 


for  all  t  €  [t„,tj]  with  h  6  H((e/e0)D().(€/€())A(l). 


Remark  2.6:  Reference  and  disturbance  signals  are  not  required  to  be  persistently  exciting  (PE). 

□ 

Remark  2.7:  It  can  be  shown  that  if  w(t)  and  f(t.0,x)  are  periodic  (almost  periodic)  in  t.  then 
h(t.0:e)  is  periodic  (almost  periodic)  in  t. 


Example  2.2:  Returning  to  Example  2.1,  we  now  use  Corollary  2.1  to  prove  that  on  the  segment 
0  =  [2.25.3.75]  the  adaptive  system  (2.26M2.27)  possesses  an  e-l'amily  of  local  slow  manifolds 
with  h  €  H((). 0456. 4.616),  Ve  €  [0,0.2].  The  A  estimate  4.61e  is  conservative  due  to  the  fact  that 
a, |  =  2.85  is  evaluated  at  0  =  2.25,  whereas  a  =  102  is  evaluated  at  0  =  3.75.  Less  conservative 
estimates  would  result  f.  im  a  smaller  segment  0.  In  this  case  Lemma  2.1  is  trivially  satisfied  and 
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K  «  1  implies  p3( D)  =  0.  The  Assumptions  2.2  and  2.3  are  satisfied  by 

v  =  0.6.  Vj  =  0.48.  v2  =  0.77, 
pF(D)  =  0.242  +  2.52D  +  5.23D2.  pz(D)  =  3.64  +  512D2. 
p«(D)  =  0.704  +  4.37D  +  5.98D2. 

and  p^D.A)  and  p2(D,A)  are  taken  to  be 

Pi(D.A)  =  0.704  +  4.37D  +  5.98D2  +  (2.52  +  10.5D)A. 
p,(D.A)  =  0.524  +  4.04D  +  6.89D2  +  (1.21  +  5.04D)A. 

These  expressions  are  valid  for  both  cases,  constant  input  r=»l  and  periodic  input  r  =  coswt.  and  are 
used  to  show  that  we  can  take  D0  =  .009  .  At>  =  0.922  and  e0  =  0.2  in  Corollary  2.1. 


Before  considering  in  the  next  section  the  attractivity  of  Me  and  sufficient  conditions  for  the 
stability  or  boundedness  of  solutions  of  (2.36)-(2.37).  we  give  an  instability  result.  The  evolution 
of  (2.36M2.37)  restricted  to  Me  is  governed  by  the  reduced-order  system 

9  =  €F(t.p.(0).h(t.0:€)) .  (2.57) 

The  next  theorem  follows  from  the  definitions  of  integral  manifold  and  instability  (in  the  sense  of 
Lyapunov). 

Theorem  2.2  :  Suppose  that  the  conclusion  of  Theorem  2.1  holds  and  that  e  €  (().€»].  Let  9  (t)  be 

any  solution  of  (2.57)  which  is  bounded  for  finite  time.  If  0’(t)  is  an  unstable  solution  of  (2.57). 

then  z’(t)  =  h(t.0'(t):€)  .  0*(t)  is  an  unstable  solution  of  (2.36M2.37). 

Proof  :  Because  9'  is  an  unstable  solution  of  (2.57)  there  exists  a  p  >  0  and  t„  €  R  such  that  for 
each  8  >  0  there  exists  a  solution  0j,(t)  of  (2.57)  with  1 0*(to)  —  0/,(t(l)  I  <  8  and 
0'(  if,)  —  t;.)  I  ^  p  for  some  tj,  ^  t„.  l  or  the  same  p  >  0  and  t„  €  R  and  each  8  >  0.  the 

solutions  \>. i,  t)  =  a  and  X  (l)  =  of  (2.36 )-(2. 37)  satisfy 

v?./(  l  +A)' 1  >  u  l  u 


1  Xj./I  |*^|(  t(i)  —  \  (  t.) )  1  <8.  I  X_y(  l+^)(t^/(  1+^))  —  X  (t^/(l+_\))  I  ^P- 


(2.58) 


,«r  J  * 

£ 

where  the  second  inequality  follows  from  I  [zT  0T]  1^101 


Letting  B(8,0)  denote  the  ball  of  radius  8  centered  at  0, 

B(8.0)  =  {0:  10-01  <  8>  , 

we  translate  this  instability  result  to  the  original  system. 

Corollary  2.2  :  Suppose  that  the  hypotheses  of  Theorem  2.2  are  satisfied  and  that  B(2p.0*(t))  C  9 
for  all  t  ^  t0.  Then,  for  t  ^  t0  ,  x’(t)  =  g(t.0*(t):€)  .  0*(t)  is  an  unstable  solution  of  (2.1M2.2). 


2.4.  Attractivity  of  the  Slow  Manifold 

While  the  existence  of  an  integral  manifold  is  sufficient  to  show  that  unstable  solutions  of 
(2.57)  combined  with  z  =  h(t.0;e)  provide  unstable  solutions  of  (2.36)-(2.37),  existence  alone  is  not 
sufficient  to  show  that  stable  solutions  of  (2.57)  lead  to  stable  solutions  of  (2.36M2.37).  In  this 
section  we  derive  conditions  under  which  Mt  is  exponentially  attractive  and  give  an  estimate  of  the 
region  of  attraction.  We  then  give  examples  of  how  this  result  is  used  to  prove  that  the  stability 
properties  of  a  solution  of  (2.57)  are  also  the  stability  properties  of  the  corresponding  solution  of 
(2.36M2.37). 

Lemma  2.3  :  Suppose  that  Assumptions  2.1.  2.2.  and  2.3  hold  and  let  e.  D,  and  A  be  such  that 
\1  =  {t.0.z  :  7.  =  h(t.0:€)}  with  h  6  H(D.A)  is  an  integral  manifold  of  (2.36)-(2.37).  Then  for  every 
Di  >  (K  +  1  )D  such  that 


ek(v,  +  A)pz(D!)  <  a,(€.D,) 


(2.59) 


the  solutions  z(t).  0(t)  of  the  modified  system  (2.36)-(2.37)  starting  from  any  bounded 
0(t,,)  =  9„  €  R"*  and  any  z(t,,)  =  z„  bounded  by 


!zJ* 


(2.60) 


satisf  v 


|z(t)-h(t.0(t);€)|  ^  K  |  Zo  —  h(to,0o;€)  I 


(2.61) 


for  all  t  ^  t0  and  any  t0  €  R.  where  a^e.D,)  =  ajCe.Dj)  -  eK(v,  +  A)p2(Dj) . 

Proof:  Suppose  that  I  z(t)  |  ^  Dj  for  all  t  ^  to-  By  Lemma  2.1 

5  =  A(0(t))z  (2.62) 

is  exponentially  stable  with  constant  K  and  rate  orj(€,Dj).  A  converse  Lyapunov  theorem  from 
Yoshizawa  (1966.  p.  90)  shows  that  there  exists  a  Lyapunov  function  V(t.z)  satisfying 

Iz!<  V(ti)  ^  K|z|.  |V(t.z)- V(t.z)|  <  K|z-z|  (2.63) 

v(2.36)(l-z)  ^  ~  Q!,(€.Di)  V(t.z).  t  ^  t0  ,  (2.64) 

where  V(236)  is  the  upper  right  derivative  of  V  along  solutions  of  (2.36).  Because  V(t.z)  is 
Lipschitzian  in  z.  h  is  Lipschitzian  in  0.  and  0(t)  is  Lipschitzian  in  t,  V(t,z  —  h(t.0;e))  is  a 
continuous  function  of  t  along  the  solutions  of  (2.36)-(2.37).  In  order  to  evaluate 

^<2 ,36>-<2  37)(t.z— h(t.0:e))  it  is  helpful  to  first  determine  expressions  for  z(t+A)  given  z(t)  =  z  and 
h(t  +  A.0(t+A):e)  given  0(t)  =  0: 

z(t+A)  =  z  +  A[A(p(0))z  -  €i'6(t.p(0))F(t,p(0).z))]  +  0(  A2)  .  (2.65) 

h(i+A.0(t+A):e)  =  h(t+A,0  +  A€F(t.p(0).h(t.0;€)):e) 

+  h(t+A.0(t+A);e)  —  h(t+A,0  +  A€F(t.p(0).h(t.0;€));e) 

=  hr t.9:e)  +  A[A(p(0)))h(t.0:€)  -  e  i/9(t.p(0))F(t.p(0)).h(t.0:€))]  (2.66) 

+  [h(t+A.0  +  AeF(t,0,z):e)  —  h(t+A.0  +  eF(t.0.h(t.0;€):€)] 

+  ()( A2) . 

From  these  expressions  and  (2.63M2.64)  it  follows  that  V(t.z  -  h(t.0:€))  satisfies 


V<2 36)— <2.37)(t'z  —  h(t.0;€))  ^  — a1(e.D1)V(t,z  —  h(t,0;e)) 


+  K  €( vj+A)  I  F(t,p(0)).z)  —  F(t,p(0)).h(t.0;€))  I 
< -a1(e.D1)V(t.z  -  h(t.0;€))  (2.67) 

+  €  K(v1+A)pz(D1)  I  z  —  h(t.0:€)  I 
^  — a2(€,Dt)V(t.z  —  h(t.0:e)) 

for  all  t  ^  t0.  which  in  view  of  (2.63)  proves  (2.61)  for  all  t  ^  t0.  This  argument,  conditioned  on 
the  assumption  that  |z(t)|  ^  D,  for  all  t  ^  t0.  also  proves  that  (2.61)  holds  for  t  €  [t0.ti]  if 
I  z(t)  I  ^  Di  on  this  interval.  The  proof  that  tt  =  °o  is  by  contradiction  .  Assume  that  there  exists 
t.  €  [t„.oo)  such  that  |z(t.)|  ^  D]  and  let  t!  be  the  smallest  such  time.  Since  |z(t0)|  <  Dj,  tj  >  t0 
and 


I  z( 1 1 )  I  ^  I  h(t1.0(t,):€)  |  +  |  z(t,)  —  h(t,.0(ti):e)  |  <  D  +  K|  z«,  —  h(t„.0o)  |  <  D,. 
which  contradicts  I  zftj  |  ^ 

Remark  2.8:  If  e  and  D1  satisfy 


(2.68) 

□ 


€  KviPf(Di)/<*i(€.Di)  <  D]  .  (2.69) 

then  (2.60)  can  be  relaxed  to  k  I  z0  I  <  D,  when  K  >  1  or  I  I  <  Dj  when  K  =  1.  See  Lemma  3.5 
in  Chapter  3. 

□ 

With  this  remark  in  mind  we  summarize  the  existence  and  attractivity  of  M(  in  the  following 
theorem. 

Theorem  2.3:  Suppose  that  Assumptions  2.1.  2.2,  and  2.3  hold.  Given  any  fixed 
A,,  >  0  .  D«,  >  0  .  I),  >  D(1  ,  and  a  €  (O.ar,,).  there  exists  e,  >  0  such  that  for  each  e  6  [O.e,]  there 
exists  h  €  H((€/€j )D,,.(€/6] )A,|)  such  that  \le  —  lt.0.z  :  z  =  h(t,0.€)}  is  an  integral  manifold  of 
(2.36)-(2.37),  and  furthermore,  solutions  z(l).  0(t)  of  (2.36 )-( 2.37 )  with  z(t,,)  =  z,,  and 
K  1  z,,  I  <  D,  satisfy  (2.61 )  with  <*;.(€. D, )  ^  a. 

Proof:  Choose  e,,  as  for  Theorem  2.1  Then  choose  e,  ^  €,,  such  that  <*-(€,. I),)  ^  a.  Thus.  €,  is 


chosen  to  satisfy  the  most  restrictive  of  five  inequalities;  hence,  it  is  easily  computed  given  the 
functions  p. 

□ 

Similarly  to  Theorem  2.1  this  result  can  be  translated  to  the  original  system  (2.1)-(2.2). 

Corollary  23:  Suppose  that  Assumptions  2.1.  2.2.  and  2.3  hold.  Let  x(t).  0(0  be  the  solution  of 
(2.1 )-( 2.2 )  with  initial  data  x(to)  =  x0 . 0(to)  =  0O  €  0.  Suppose  that  0(0  €  0  for  all  t  €  [to-tj]- 
Given  any  fixed  A0  >  0  .  D0  >  0  .  Dj  >  D0.  and  a  €  (0,a0).  there  exists  >  0  such  that  for  each 
e  €  [0.€,]  there  exists  h  €  H((6/€,)D0  .  (e/e^Ao)  with  the  following  properties: 

i)  if  (to.0„.xo)  €  then  (t,0(t).x(t))  6  M(  for  all  t  €  [t0-tI ]. 

ii)  if  K  I  x„— v(to,0o)  I  <  Dj.  then  for  all  t  6  [t„.t,] 

I  x(t)  —  g(t,0(t);  €)  I  <  Ke_“<l_t°)  I  x0  —  g(to.0o^)  I  •  (2.70) 

Theorem  2.3  suggests  that  solutions  of  the  modified  system  (2.36)-(2.37)  have  two  "time- 
decoupled"  parts,  one  being  rapid  convergence  to  the  slow  manifold  M,  and  the  other  being 
evolution  near  M*.  Motivated  by  this  observation  we  rewrite  (2.37)  in  the  form  of  (2.57)  with  a 
rapidly  exponentially  decaying  perturbation. 

0  =  €F(t.p(0).h(t.0:e))  +  €[  F(t,p(0).z)  —  F(t.p(0).h(t.0:€))  ].  (2.71) 

and  use  it  to  show  that  the  stability  properties  of  a  solution  0(t)  of  (2.57)  are  inherited  by  the 
solution  0(t).  z(t)  =  h(t.0(t):€)  of  (2.36)-(2.37).  We  show  this  for  the  case  of  a  uniformly  stable 
solution  0’  of  (3.25).  The  first  step  is  to  recall  a  converse  Lyapunov  theorem;  see.  for  example. 
Yoshizawa  ( 1966). 

Lemma  2.4:  Suppose  that  0‘(l)  is  a  uniformly  stable  solution  of  (2.57).  that  Blk^O’ft))  £  0  for  all 
t  €  R.  and  that  f ( t .0.x )  is  a  continuous  function  of  t.  Then,  there  exist  k2  €  (O.Kj).  a  Lyapunov 
function  I .( t.0 ).  two  strictly  increasing  positive  functions  y\  and  y2.  and  a  constant  l  .  such  that, 
lor  t  ^ t,,.  0—0*1  ^  k2.  and  I  0—0*  I  ^  k2. 

y,(  |  0  —  0*  I  )  ^  L(t.0  —  0’)  ^  y2(  l  0  —  0’  I  )  .  y1(0)  =  y2(0,  =  0  (2.72) 
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* 

satisfies  W(2  36M2  37)(t.0.z)  Oif|z|<  Djand|0  —  0’ |  ^  K2.  This  proves  the  following  theorem. 
Theorem  2.4:  Suppose  that  the  conclusions  of  Lemma  2.4  and  Theorem  2.3  hold  and  that  €  6  (O.eJ. 
Choose  k3  <  0  such  that  y,(  101  )  <  K3  implies  101  <  K2.  Then,  the  solution  of  (2.36M2.37) 

with  initial  data  z(t„)  =  z„  .  0(t«)  =  0 „  for  any  (to.0o.z())  in  the  set 

So  =  It.d.z  :  K  I  z  I  <  D,  .  W(t.0,z)  <  K3}  .  (2.7S) 

remains  in  the  set  Sj  =  !t.0.z:  Izl  <  D!  .  W(t.0.z)  <  K3}  for  all  t  ^  t0  and  along  each  solution 

^ (.2  37>(t.0.z)  <  0.  Moreover,  for  every  constant  S  >  0.  there  exists  t6  >  0  such  that 

i  t.0(  l ).'/.( t )}  €  Sf  for  a  1 1  t  ^  t„  +  t;,  where 

S,s  =  |t.0.z  :  z  -  h(t.0;e)  I  <  5  .  L(t.0-0*(t))  ^  \V(tu.0„.z,v)|  .  (2.79) 

□ 

this  theorem  shows  that  the  umtorm  stability  of  0  (t)  as  a  solution  of  (2.57)  combined  with  the 
existence  ot  an  exponentially  attractive  integral  manifold  Me  implies  that  z*(t)  =  h(t,0’(t)'.e).  0’(t) 
is  a  uniformly  stable  solution  of  (2.36)-(2.37).  Since  (t,0.z)  €  S,  implies  that  0  €  0.  Theorem  2.4 
can  be  translated  to  the  original  system  (2.1 )-( 2.2 ). 
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Corollary  2.4  :  Suppose  that  the  hypotheses  of  Theorem  2.4  hold.  Then,  the  solution  of  (2.1)-(2.2) 
with  initial  data  x(t0)  =  x0 . 0(to)  =  0O  for  any  to.0o.xo  in  the  set 


So  =  |t.0.x  :  K  I  x  -  j/(t,0) !  <  D,  .  W(t.0.x-i/(t.0))  <  K3} 


(2.80) 


remains  in  the  set  St  =  (t,0,x  :  I  x— v(t.O)  I  <  Dj  .  W(t.0.x— p(t,0))  <  K3}  for  all  t  ^  t0  and  along 
each  such  solution  W(2  j)_(2  2)(t.0.x— t/(t,0))  ^  0.  Moreover,  for  every  constant  8  >  0.  there  exists 
tf,  >  0  such  that 


S6  =  {t.0.x  :  I  x— g(t,0:e)  I  <  8  .  L(t.0-0*(t))  ^  W(to.0o.xo)-j/(t„.0o)} 


(2.81) 

□ 


Remark  2.9:  Whereas  results  similar  to  those  of  Theorem  2.4  are  usually  obtained  under  the  more 
restrictive  assumption  of  the  uniform  asymptotic  stability  of  0*(t)  as  a  solution  of  (2.57).  in 
Theorem  2.4  0'(t)  is  only  assumed  to  be  uniformly  stable.  This  stronger  result  which  is  due  to  the 
fact  that  O'  is  a  solution  of  the  exact  manifold  equation  (2.57).  rather  than  an  approximation,  is  of 
conceptual  interest.  The  hypothesis  of  Lemma  2.4  may  not  be  verifiable  which  limits  the  direct 
applicability  of  the  result. 


□ 


Remark  2.10:  The  same  proof  technique  can  be  applied  to  show  that  uniform  asymptotic  stability 
or  exponential  stability  of  0*(t)  implies  uniform  asymptotic  stability  or  exponential  stability, 
respectively .  of  x‘(t)  =  g(i.0*(l);e),  0’(t).  Similar  techniques  apply  when  0’(t)  is  replaced  by  an 
invariant  set.  The  use  of  a  quadratic  Lyapunov  function  in  place  of  W  may  provide  better 
estimates  of  the  region  of  au.  action;  see  Saberi  and  Khalil  (1981 ). 


□ 


Remark  2.11:  It  may  not  be  necessary  to  find  the  manifold  function  h(t,0:e)  in  order  to  determine 
a  solution  0  (t)  of  (2.5  7).  Lor  example,  if  ( 2.36 )-( 2.37 )  has  an  equilibrium  at  z  =  0 .0  =  0°.  then 
0’(t )  =  0P  is  a  solution  of  (2.57). 


In  specific  case  studies  a  more  elaborate  construction  can  lead  to  an  estimate  of  the  domain  ol 
attraction  less  conservative  than  S,,  in  Theorem  2.4 

Example  2.3:  Consider  again  Example  2.1  with  constant  r=l.  y =  0.5  Let  0,  be  a  segment 
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[\,  .  A,]  £  0.  Similarly  to  Lemma  2.3  we  can  show  that  if  I  z  |  ^  Dj  and  0  €  ©,.  then  the  derivative 
of  V(z  —  h(0):e)  =  I  z  —  h(0;€)  |  satisfies 


V(2.36M2.37)(Z— ^ h(0;e))  <  — 


\,-l-  e 

4.  A 

3— A,  2D+D. 

+ 

(Xi-1)2 

2(A,— 1 )  1-mA, 

V(z-h(0;e)) 


(2.82) 


and  the  derivative  of  L(0)  =  1 0  —  3  |  satisfies 


L(2.36M2.37)(0)  ^  L(2.57)(0 )  +  e 


3  -  A, 


2D  +  D, 


UK-  1)  i  - MA. 


V(z-h(0;€)) 

1  —  /x0 


(2.83) 


For  small  enough  e  the  right-hand  side  of  (2.82)  is  negative.  We  use  /tt  =  .25.  €  =  .1.  Taking  W  as 

t  i 

an  appropriate  linear  combination  of  V  and  L.  we  can  achieve  W(2  36)_(2  37)(0.z)  ^  L(2  57)(0)  <  0  if 
L(0)  ^  .09.  Choosing  different  linear  combinations  of  V  and  L  for  different  subsets  of  0,  of  0  we 
create  a  comparison  function 

W(0.z)  =  -~-r-  (m;L(0)  +  V(z  —  h(0:e))  +  y,)  if0  €  0,  (2.84) 

3.66 


with  m,.  yv  and  0j  listed  in  Table  2.1.  The  constants  y,  are  chosen  so  that  W(0.z)  =  C],  a  constant, 
is  the  boundary  of  a  compact  set.  This  construction  is  such  that  if  Cj  <  c2.  then 
(0.z  :  W(0.z)  ^  c(l  C  {0.z  :  W(0.z)  <  c2|.  The  m,'s  are  chosen  so  that 

W,2  ■(h)_(2 37)(0.z)  ^  Ll257)(0)  <  0  if  L(0)  ^  .09  and  m,  =  0  if  L(0)  ^  .09  .  Thus  any  solution  of 
( 2.36 )-( 2.37)  with  (0,,.^,)  €  S(,  =  |0,z  :  W(0.z)  ^  0.75|  remains  in  Sn  for  all  t  ^  t(,  and  converges  in 

Table  2.1.  Parameters  of  W(0.z) 


lil 

Q-, 

0, 

m^m^, 

V,=V-. 

1 

[2.91.300] 

[3.00.3.09] 

0.00 

1.143 

2 

[2.85.2.91] 

[3.09.3.15] 

1.80 

0.981 

3 

[2.75.2.85] 

[3  15.3.25] 

1.92 

0.963 

[2.65.2.75] 

[3.25.3.35] 

2.04 

0.93.3 

5 

[2.55.2.65  j 

[3.35.3.45] 

2.20 

0.877 

6 

[2.45.2.55] 

[3.45.3.55] 

2.39 

0.7914 

7 

[2.35.2.45] 

[3.55.3.65] 

2.73 

0.6045 

* 

[2.25.2.35] 

[3.65.3.75] 

3.66 

0.00 

i 


finite  time  to  S6  =  ( 0.z  :  1 0  —  3  |  <  0.09.  |  z  —  h(0:e)  |  <  8).  By  construction  (0.z)  €  S„  =>  0  €  0; 
hence,  these  solutions  can  be  related  to  solutions  of  (2.26 )-( 2.27 )  with  x  =  v{6)  +  z  as  in  Corollary 
2.4.  The  trajectories  in  Fig.  2.3  which  begin  from  the  vertices  of  the  polygon  W(0,x— v(0))  =  0.75 
clearly  show  that  the  equilibrium  0  =  3.  x  =  i>(3)  =  0.375  has  a  domain  of  attraction  containing  S(). 
For  0^3  the  trajectories  cross  the  boundary  of  S0  almost  perpendicularly  indicating  that  in  this 


region  S<,  is  a  conservative  estimate  of  the  domain  of  attraction.  However,  this  estimate  is  designed 
to  guarantee  that  0(t)  ^  3.75  for  all  t  ^  t„,  and  is  much  less  conservative  for  0  >  3.  This  can  be 
seen  from  the  trajectory  (a)  converging  to  the  manifold  and  the  close-by  divergent  trajectory  (b). 


2-5.  Attractive  Integral  Manifolds  of  a  Model  Reference  Adaptive  Control  System 

In  this  section  we  put  a  model  reference  adaptive  control  system  (MRAS)  in  the  form  of 
(2.1 )-( 2.2).  Then,  in  order  to  show  that  the  MRAS  possesses  a  slow  manifold,  it  is  sufficient  to 
show  that  Assumptions  2.1. 2.2.  and  2.3  are  not  restrictive. 

As  the  first  two  assumptions  are  concerned  only  with  properties  of  (2.1)  for  constant  values 
of  0.  we  postpone  the  specification  of  the  parameter  update  law  (2.2)  until  later.  The  controller 
parametrization  of  \arendra  and  Valavani  (1978)  is  common  to  several  MRASs;  a  block  diagram  of 
the  controlled  system  is  shown  in  Fig.  2.4.  Assuming,  for  ease  of  exposition,  that  the  plant  is 
strictly  proper,  the  controlled  system  is  described  by  (2.1)  with 


A(0)  =  A,,  +  b‘0T 


A„  = 


,  B(0)  =  [0„  b1  I  bJ  I  0]b*  +  b2] 


AP 

i  i 

.  b1  = 

bp 

0 

br 

0 

hr 

.  b2  = 

0 

.  bJ  = 

0 

bfh,, 

0 

0 

(2.85) 


h,T 

Xp 

C  = 

0 

.  X  = 

V1 

0 

-> 

w(t)  = 


r(l) 
nt(  t) 
n-.(  t ) 


0  = 


where  r  is  the  reference  input,  and  n|  .  n;  are  disturbances  and  without  loss  ol  generality 
h  =1.  We  get  the  \arendra  and  Valavani  controller  designed  lor  a  plant  of  relative  degree  one 
and  order  n  bv  assigning  the  dimensions  c,,  €  R  .  cl,,  €  R  .  c  6  R"_l  .  d  €  R 


n-i  a  a  i?‘i-i  v1  6  R'1”1  .  v2  6  R"-1 


The  trajectories  of  (2.26)-2.27)  beginning  at  the  vertices  of  W(0.x— 1/(0))  =  0.75  converge 
to  the  equilibrium.  Along  trajectory  (a)  0  leaves  the  s'et  0  =  [  2.25  3.75  ].  but  con¬ 
verges  to  the  manifold,  and  then,  to  the  equilibrium  Initially  nearby  trajectory  (b)  is 
divergent. 


n,(l)  n2(t)  —  y„,(t) 


dr 

Block  diagram  ol  the  \arendra  and  Yalavani  (  1  *>7K )  controller  purametrixation 
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and  choosing  the  eigenvalues  of  A  to  be  the  zeros  of  the  reference  model.  In  order  to  show  that  an 
integral  manifold  exists  we  require  A.  B.  and  w  to  satisfy  Assumptions  2.1  and  2.2. 

Assumption  2.1  implies  the  stabilizability  of  the  unknown  plant  by  the  chosen  controller.  In 
order  to  see  that  it  is  not  very  restrictive  we  state  a  proposition. 


Proposition  2.1:  If  there  exist  0"  €  and  a0  >  0  such  that 

Re  A(A(00))  <  -«o  (2  86) 

then  Assumption  2. 1  is  satisfied. 

/ 

Proof  from  (2.86)  it  follows  that  exp[t.A(0o)]  ^  K  exp[— o(,t]  for  some  K  <  oo.  From  standard 
arguments  about  the  stability  of  perturbed  linear  systems,  it  follows  that  Assumption  2.1  is 

satisfied  with  a,,  =  a„/2  and  0  =  {0  €  RM  :  I  0—0°  i  ^  K  I  b1  I  /(2a„)). 

□ 

Thus  our  stabilizability  assumption  does  not  require  the  knowledge  of  the  plant  order,  nor 
does  it  require  the  plant  to  match  a  reference  model.  We  shall  take  the  hypothesis  of  Proposition 
2.1  as  a  hypothesis  of  Theorem  2.5.  However.  a„  .  K  .  and  ©  are  important  quantities  in  the 
analysis  of  the  previous  sections  and  estimating  them  via  the  proof  of  Proposition  2.1  is,  in  general, 
very  conservative.  We  suggest  that  they  be  estimated  in  an  off-line  analysis  via  analytic 
expressions,  simulation,  or  experimentation. 

The  following  proposition  shows  that  Assumption  2.2  is  not  restrictive. 

Proposition  2.2:  If  Assumption  21  holds,  and  w(t)  is  uniformly  bounded,  piecewise  Lipschitz 
continuous,  and  there  exists  5  >  0  such  that  all  points  of  discontinuity  are  separated  by  at  least  5. 
then  Assumption  2.2  is  satisfied. 


V 


Proof.  The  boundedness  and  regularity  of  w(t)  combined  with  Assumption  2.1  are  sufficient  to 
guarantee  that  vft.  0)  is  a  lipschitz  continuous  function  of 'time,  letting  11  r  11  =  sup  lr(t)l  we  can 


33 


compute  bounds  on  v.  vj  .  and  v2  as 


v  ^  —  sug  II  0oblr  +  b3nj  +  (djb1  +  b2)n2  I 

Vj  ^  —  lb1!  [  II  r  II  +  v  +  II  n^>  II  ] 

«o 

v2^2-L  lb1 1  vt  . 

«o 


(2.87) 


Thus  Assumption  2.2  is  a  regularity  and  boundedness  assumption  on  the  external  inputs  to 
the  system.  An  important  observation  concerning  the  applicability  of  this  analysis  in  the  design  of 
adaptive  control  schemes  is  that  these  assumptions  are  stated  for  the  system  with  constant 
parameters.  These  are  assumptions  about  the  chosen  controller  structure  and  the  signals  expected 
to  enter  the  controlled  system. 

While  the  previous  two  assumptions  were  independent  of  the  parameter  update  law.  the  last 
assumption  depends  only  upon  the  parameter  update  law.  For  the  MRAS  which  we  are  considering, 
the  update  law  is  given  by  (2.2)  with 


f(t,0.x)  =  f(t.x)  =  — 


r  0 

+  n2  (hpTxp  +  n2  -  yni(t)) 
Cx  o 


(2.88) 


where 


r,(t)  =  f  hj  exp[Am(t-r)]bmr(T)dr 


(2.89) 


An,  €  Rn:<n  is  Hurwitz.  and  Wm(s)  =  h^(sl— Am)  ‘bm  is  strictly  positive  real  (SPR).  The  update  law 
must  satisfy  Assumption  2.3.  It  is  straightforward  to  establish  that  I  x  I  ^  I)  implies  that 


f(t.x)  I  ^  pF(D)  =  (  II  r  II  +  II  n2  II  +  D)(  II  n2-vm  il  +  D)  . 


(2.90) 


(t.x)  I  ^  px(D)  =  H  r  II  +  II  n-.  II  +  il  n->— vm  II  +  21) 

ax 


(2.91) 


Then,  under  Assumption  2.2.  Assumption  2.3  is  satisfied  with 
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a 


Pf(D)  ^  pf(v+D)  .  Pj(D)  ^  p„(v+D)  .  Pq(D)  <  v,px(v+D) 


(2.92) 


Hence,  we  have  the  following  result  concerning  the  existence  of  an  attractive  integral  manifold  Mt 
for  the  MRAS  described  by  (2.1)-(2.2).  (2.85),  (2.88M2.89). 

Theorem  2-5:  Suppose  that  the  hypotheses  of  Propositions  2.1  and  2.2  hold.  Then,  with  A.B.  w. 
and  f  given  by  (2.85).  (2.88)-(2.89).  the  conclusions  of  Theorem  2.3  hold  for  the  modified  system 
(2.34M2.35)  and  the  conclusions  of  Corollary  2.3  hold  for  (2.1  )-(2.2). 

□ 

Remark  2.12:  To  this  point  we  have  made  no  assumptions  about  persistent  excitation,  sufficient 
richness,  periodicity  or  almost  periodicity  of  the  signals  entering  the  adaptive  system. 

□ 

Remark  2.13:  If  w(t)  and  f(t.fl.x)  are  periodic  (almost  periodic)  in  t.  then  v(t.0)  and  h(t.0:€)  are 
periodic  (almost  periodic)  in  t. 
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2.6.  Stability  in  the  Manifold:  Averaging 

In  Sections  2.3  and  2.4  we  derived  conditions  for  the  existence  of  an  attractive  local  integral 
manifold  Mf  of  (2.1  )-(2.2 )  and  showed  that  the  stability  properties  of  a  solution  0(t)  of 


0  =  ef(t.0,g(t.0:€)) 


(2.93) 


which  remains  in  0  for  all  t  ^  t<,  are  also  the  stability  properties  of  the  solution 
x(t)  =  g(t.0(t);€)  .  0(t)  of  (2.1M2.2).  We  have  established  these  conclusions  without  recourse  to  an 
averaging  argument.  However,  equation  (2.93)  is  in  the  standard  Bogoliubov  form  for  the  method 
of  averaging.  Hale  (1980).  Meerkov  (1973),  Sethna  and  Moran  (1968),  Volosov  (1962),  Bogoliubov 
and  Mitropolski  (1961).  Although  averaging  is  not  the  only  means  for  analyzing  (2.93),  it  is  the 
one  which  we  shall  apply.  The  method  of  averaging  gives  very  strong  results  for  (2.93)  in  general, 
and  especially,  when  (2.1 )-( 2.2 )  represent  an  adaptive  system  as  in  Section  2.5.  including 


spun! 
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(i)  analysis  with  enough  precision  to  provide  sufficient  conditions  for  instability  as  well  as 
sufficient  conditions  for  stability,  and 

(ii)  interpretation  in  terms  of  the  frequency  spectrum  of  the  signals  entering  the  system  and 
certain  transfer  functions  in  the  adaptive  system. 

The  method  of  averaging  relates  solutions  of  (2.93)  to  solutions  of 

—  =1(9)  (2.94) 

dr 

where  r  =  e(t— 10)  is  the  slow  time  scale  and  using  g(t.0:O)  =  v(i .0).  f(0)  is  the  average  of  f  with  9 
constant  defined  by 

t+T 

f(0)  =  lim  —  f  f(s.0.v(s.0))ds  .  (2.95) 

T-oo  T 

We  assume  w(t)  and  f(t.0.x)  are  almost  periodic  in  t  so  that  the  limit  in  (2.95)  exists  uniformly 
with  respect  to  t.  Because  (2.94)  is  time-invariant  and  independent  of  6.  it  is  easier  to  study  both 
analytically  and  experimentally  via  computer  simulation,  than  (2.93). 

Three  theorems  from  the  method  of  averaging  lead  to  immediate  and  useful  results  for 
adaptive  systems.  One  theorem  gives  conditions  under  which  the  existence  of  a  constant  solution  of 
(2.94)  implies  the  existence  of  an  almost  periodic  solution  to  (2.93).  This  theorem  also  relates  the 
stability  or  instability  of  the  constant  solution  of  (2.94)  to  the  almost  periodic  solution  of  (2.93). 
It  is  the  essential  part  of  the  theorem  used  to  establish  a  stability-instability  criterion  for  adaptive 
systems  in  Riedle  and  kokotovic  (1985)  and  kokotovic.  Riedle.  and  Pralv  (1985).  The  other  two 
theorems  give  conditions  under  which  tne  solution  of  (2.94)  approximates  the  solution  of  (2.93). 
The  first  of  these  applies  on  a  finite  time  interval;hence.  it  applies  when  solutions  of  (2.93)  or 
(2.94)  leave  0  in  finite  time  This  result  was  used  by  Astrom  (1983.1984)  in  his  explanation  of  the 
instability  mechanisms  in  a  model  reference  adaptive  control  system.  The  second  approximation 
theorem  applies  on  infinite  intervals.  We  shall  use  it  to  provide  sufficient  conditions  for  the 
unilorm  asymptotic  stability  of  an  almost  periodic  solution  of  (2.93)  and  to  provide  an  estimate  ol 


the  region  of  attraction  which  is  not  restricted  by  linearization. 

We  consider  first  the  finite  interval  approximation  theorem 
Theorem  2.6:  Suppose  that  Assumptions  2.1,  2.2.  and  2.3  hold.  If  the  solution U(t)  of  (2.94)  with 
?(0)  =  0(to)  and  its  ©"-neighborhood  are  in  0  for  all  t  €  [0,7"!),  that  is.  if 

B (o-SCr))  C  0  ¥  r  €  [O.Tj)  (2.96) 

for  any  r(  €  (O.oo)  and  any  <J  >  0,  then  there  exists  e.(Ti,cr)  6  (0.€j]  such  that  for  each  6  €  (0,€.) 

I0(t)  -  W(€(t— t0))l  <  cr  V  t  €  [t0.to  +  ^i/«)  •  (2.97) 

□ 

Let  0 1  C  0  be  a  set  and  (rl  >  0  a  constant  such  that  0  €  ©j  implies  B(O"!.0)  Q  0. 

Corollary  2.5:  Suppose  that  Assumptions  2.1.  2.2.  and  2.3  hold  and  that  every  solution  of  (2.94) 
with  5T(0)  €  0i  £  01  leaves  0,  before  Tj.  Then  there  exists  €.  €  (O.eJ  such  that  for  each  €  €  (0.6.) 
and  for  any  t()  €  R.  every  solution  of  (2.93)  with  0(t„)  €  ©2  leaves  ©i  before  t„  +  T ]/e. 

□ 

Remark  2.14:  The  phrase  "solution  of  (2.93)  with  0(to)  6  ©2"  can  of  course  be  replaced  by 
"solution  of  (2.1M2.2)  with  0(to)  €  02  and  x(t„)  =  g(to.0(t(J):e)."  Using  the  exponential  decay  of 
I  x  —  g  I  one  can  modify  the  proof  of  Theorem  2.6  to  show  that  off-manifold  initial  values  of  x  are 
also  allowed. 

□ 

If  the  averaged  system  (2.94)  has  an  instability  mechanism  which  causes  its  solutions  to 
escape  in  finite  lime  from  ©t.  then  this  result  shows  that  an  explanation  of  the  instability  of  (2.94) 
is  a  valid  explanation  of  the  predicted  instability  of  the  original  system  (2.93)  for  sufficiently  slow 
adaptation. 

Rather  than  repeat  more  of  the  general  averaging  theorems  which  apply  to  (2.93)  and  can  be 
found  in  the  cited  literature,  we  illustrate  the  use  of  averaging  theorems  in  the  analysis  of  the 
particular  MRAS  presented  in  Section  2.5.  We  first  use  the  theorem  on  existence  and  stability  of 
almost  periodic  solutions  of  (2.93)  to  strengthen,  bv  a  precise  definition  of  the  tuned  parameter,  our 
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stability  criterion.  The  weakness  of  the  criterion  was  the  arbitrariness  of  the  choice  of  the  tuned 
value  of  the  parameter  0  around  which  the  system  (2.1)-(2.2)  was  linearized  in  Riedle  and 
Kokotovic  (1985)  and  kokotovic.  Riedle,  and  Praly  (1985).  In  terms  of  the  update  law  (2.14),  the 
function  f  in  (2.93)  is 

f(tMt.O))  =  — 0(t.0)e(t.0)  (2.98) 

where  0(t.0)  and  e(t.0)  are.  respectively,  the  values  of  the  regressor  vector  and  the  tracking  error 
for  constant  values  of  9: 

4>(i.9)  =  .  e(t.0)  =  [hpT  0  O]i/(t,0)  +  n2(t)-ym(t) .  (2.99) 

Using  the  regressor  vector  0(t .9)  as  the  input  to  the  transfer  function 

WCL(0\s)  =  [hpT  0  ()J(sl  -  A(W ))~1bI  ,  (2.1(H)) 


results  in  a  vector 


v(t.0.9")  =  WCL(F,s)^(t,0)  . 


(2.101) 


This  vector  is  important  because  when  W  =  9,  it  can  be  shown  that  v  is  the  sensitivity  of  e(t.0) 
with  respect  to  9.  that  is. 


e«(t.0)  —  -^-(t.0)  =  vT(t,0.0)  . 


(2.102) 


By  the  mixed  notation  t.s  in  (2.101).  we  mean  that  the  ith  component  of  v  is  the  almost  periodic 
response  of  the  closed-loop  transfer  function  WCL(0\s)  to  the  almost  periodic  ith  component  of  0- 
Letting  0"  be  an  arbitrary  "tuned"  parameter,  we  derive  the  equation  from  which  the  stability 
criterion  was  developed.  Linearizing  (2.94)  around  9”  =  9"  we  get 

=  —  !avg[0(-,0°)vT( •,0°.0°)]  +  avg[0„(\0t')e(  ■,0U)]K^— 0")  —  avg[0( -,0l’)e(  .0°)]  (2.103  ) 

dr 

where 
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avgfe(-,00)]  —  lim  1  /  e(t.0°)dt  . 

T-oo  T  ''q 

Our  previous  analysis,  (Riedle  and  Kokotovic  .1985.  and  Kokotovic.  Riedle.  and  Praly  .1985). 
neglected  the  terms  containing  e(t,0°)  and  investigated  the  linear  system 


=  -avg^-.^MC-.^.d0)]^-^0)  • 
dr 


(2.104) 


Its  equilibrium  W  =  0°  is  exponentially  stable  or  unstable  depending  on  the  eigenvalues  of  R(0°.0°) 


where 


RCAJ)  =avg[0(-.0)vT(-.0.g‘)] 


(2.105) 


The  stability  criterion  on  (2.104)  was  practical  because  the  eigenvalue  condition  on  the  matrix 
R(0°.0°)  is  easily  interpretable  in  terms  of  signals  and  transfer  functions  in  the  system  (2.1 )-( 2.2). 
Consider,  for  example,  the  case  when  0(t.0)  is  the  sum  of  a  finite  number  of  sinusoids 


0(t,0)  =  £  0(0.w)  e^1  . 

11 


(2.106) 


Because  0  is  real  valued.  oi€  SI  implies  — o>€  i l  and  0(0,— w)  is  the  complex  conjugate  of  0(0, o>). 
Then  we  compute  v(t.0.lT)  and  R(0.(T) 


v(t.0.5r)  =  £  0(0, w)  WCL(0\jti>)  e^' 

ft 


(2.107) 


R(0.(T)  =  £  0(0,— oi)  0T(0.w)  WCL(0\j<»>)  . 

ft 


•  (2.108) 


Hence,  the  matrix  R(0°.0°)  is  easily  computed  if  we  know  the  Fourier  series  representation  of 
0(t.0")  and  the  transfer  function  WCL(0°.jca).  An  interesting  sufficient  condition  for  (2.104)  to  be 
exponentially  stable  is  that  R(0°.01')  +  RT(0°,0°)  be  positive  definite,  that  is. 


0  <  R(0°.0")  +  RT(0°.0°)  =  £  'H&'.-u)  0T(0°.o»)  Re  WCL(0'\jo>)  . 

td£  il 


(2.109) 


The  condition  (2.109)  has  been  called  "signal-dependent  SPR"  (Riedle  and  Kokotovic.  1985.  and 
Kokotovic.  Riedle.  and  Praly,  1985)  because  it  resembles  the  usual  strict  positive  realness 
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requirement  that  Re  WCL(0°.jo>)  >  0  for  all  <a.  but  relaxes  this  requirement  by  incorporating 
information  about  the  signals  entering  the  adaptive  system. 

In  this  analysis  we  do  not  neglect  e(t.0°)  but  instead  assume  that  an  equilibrium  (T°  of  the 
averaged  equation  (2.94)  exists,  that  is. 

0  =7(9'°)  =  avg[0(J°)e(-,S'0)]  (2.110) 

Then  we  select  0  =  S’0  to  be  the  tuned  parameter  0°.  With  this  choice,  the  forcing  term  in  the 
linearized  equation  (2.103)  is  zero  and  (2.103)  is  the  linearization  of  the  averaged  system  (2.94) 
around  its  equilibrium.  Clearly,  the  stability  or  instability  of  this  equilibrium  is  determined  by 
the  eigenvalues  of 


f  #(0°)  =  y  =  — R(0°.0°)  -  Rj(0°) 

00 


(2.111) 


R,(0°)  =  avg[0a( -  .0°)e( -  .0°)]  .  (2.112) 

In  addition  to  the  easily  lnterpretable  matrix  R.  a  stability  criterion  for  (2.103)  must  deal  with  Rj, 
which  is  much  more  difficult  to  interpret.  Since  a  criterion  for  stability  and  instability  based  on 
the  eigenvalues  of  R  +  Rt  is  much  less  appealing  than  the  criterion  for  (2.104),  our  goal  is  to 
formulate  sufficient  conditions  for  stability  and  instability  in  terms  of  the  RMS  error 

E(0)  =  iavg[e-(  .0)]}"  .  (2.113) 

which  will  appear  in  a  bound  for  R,.  and  the  eigenvalues  of  R(0°,0°). 


« 

0 
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In  the  noncritical  case,  that  is.  when  no  eigenvalues  of  R(0“.0°)  have  zero  real  parts,  we  can  always 
find  a  transformation  T(0")  such  that 

A+  0 

T( 0  l  R , ( 0  .0" )  T_l(0’ )  =  I  T_1(0°)  1^1.  (2.114) 

<>  A_ 

sc  here  all  the  eigenvalues  ol  A„  have  positive  real  parts  and  all  the  eigenvalues  of  A_  have  negative 
real  parts  f  urthermore,  there  exist  positive  constants  m  and  A  such  that 
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Theorem  2.7:  Suppose  that  Assumptions  2.1.  2.2.  and  2.3  hold.  Let  9°  =  &°  be  an  equilibrium  of 
the  averaged  system  (2.94)  in  the  interior  of  8.  Suppose  that  no  eigenvalues  of  R(0°.0°)  have  zero 
real  parts.  Let  T(0°)  be  a  transformation  satisfying  (2.114)  and  m.A  be  constants  such  that  (2.115) 
holds.  If  the  RMS  error  (2.113)  is  bounded  by 


E(0°)  < 


A  I  T(0°)  I 
m  III  0@(-,0°) 


(2.116) 


where  <f>e  -  ^  and 


l</>H(-.e°)  III  -  {avg[  l  <£g(-.0°)  1 2]}v 


(2.117) 


then  there  exists  €.  €  (O.eJ  such  that  for  each  €  6  (0,6.].  the  original  system  (2.93)  possesses  a 
unique  almost  periodic  solution  0*(t,e)  which  tends  to  6°  as  6  tends  to  zero,  that  is. 
Um  1 0*(t.€) — 0°  I  =0.  Furthermore.  0*(t,e)  is  uniformly  asymptotically  stable  if  all  the 

eigenvalues  of  R(0°.0°)  have  positive  real  parts  (that  is.  the  dimension  of  A_  is  zero),  and  0’(t.€)  is 
unstable  if  one  eigenvalue  of  R(0°.0")  has  a  negative  real  part  (that  is.  the  dimension  of  A_  is 
greater  than  zero). 

Proof:  It  is  sufficient  to  show  that  no  eigenvalues  of  R(0‘\0°)  +  R^O^have  zero  real  parts  and  that 
the  dimension  of  A_  is  equal  to  the  number  of  eigenvalues  of  R(0O,0°)  +  R^O0)  with  negative  real 
parts.  The  conclusions  then  follow  from  averaging  theorems  such  as  Theorem  V.3.1  of  Hale 
( 19SO).  Applying  the  transformation  0  =  T(0„)((F— 0")  to  (2.103)  we  have 


—  0  = 


A+  0 


0  A_ 


+  T(0°)R  1(0,’)T-1(0‘’)  0 


(2.1  IS) 


By  the  Cauchy-Schwartz  inequality 
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IR,(0°)l  <  III  <£#(t.0°)  III  E(0°)  ^  vjEfd0)  (2.119) 

where  the  second  inequality  follows  from 

*»<■•«“>  =  0»<t.8>  •  <212°> 

ICI  =1.  and  Assumption  2.2.  Then  (2.115)-(2.116)  and  the  fact  that 

+  T(0°)R,(0o)T-l(0°)  is  similar  to  R(0°,0°)  +  R,(0°)  imply  that  no  eigenvalues  of 

f #(0°)  =  ~R(0°.0°)  —  RjCd0)  have  zero  real  parts  and  that  the  dimension  of  A_  is  equal  to  the 
number  of  eigenvalues  of  f  #(0°)  with  positive  real  parts. 

□ 

In  light  of  Corollary  2.2  and  the  results  of  Section  2.4.  we  have  the  following  corollary. 

Corollary  2.6:  Under  the  conditions  of  Theorem  2.7.  for  each  €  €  (0,€.j  the  system  (2.1)-(2.2) 
with  definitions  (2.85).  (2.88)-(2.89)  possesses  a  unique  almost  periodic  solution 

x*(t.€)  =  g(t.0*(t.€);e)  .  0*(t.€)  in  a  neighborhood  of  Kt.0°)  .  0°.  Furthermore,  this  solution  is  u.a.s.  if 
all  the  eigenvalues  of  R(0°.0°)  have  positive  real  parts  and  unstable  if  one  eigenvalue  of  R(0°.0°)  has 
a  negative  real  part. 

□ 

Although  Theorem  2.7  and  Corollary  2.6  are  also  based  on  linearization,  this  result  is  more 
complete  than  the  original  stability  criterion  results  of  Riedle  and  kokotovic  (1985)  and  Kokotovic. 
Riedle.  and  Praly  (1985).  It  is  more  complete  because  the  choice  of  the  tuned  parameter  0°  =  ?F° 
allows  the  conclusions  to  apply  to  the  actual  MRAS  (2.1 )-( 2.2).  However,  by  itsell .  the  choice  of 
the  tuned  parameter  0"  =  S’”  does  not  provide  sufficient  guidance  for  the  design  and  analysis  of  an 
MRAS.  The  existence  of  W"  is  not  obvious  and.  except  for  the  case  H(0")  —  0.  defining  the  tuned 
parameter  9"  as  the  solution  of  (2.110)  does  not  give  a  characterization  of  0°  which  is  easily 
interpreted  or  checked  in  terms  of  the  properties  of  the  controlled  system  (2.1)  with  constant  0. 


A+  o 
0  A_ 


I 


I 
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In  the  ideal  case,  no  disturbances  and  no  unmodeled  dynamics,  the  parameter  update  law 
drives  the  tracking  error  to  zero.  An  assumption  of  persistent  excitation  on  the  regressor  vector 
then  implies  that  the  function  E(0)  has  a  unique  global  minimum  with  min  E(0)  =  0.  In  the 
nonideal  case  we  are  studying,  suppose  that  the  function  E(0)  has  an  isolated  local  minimum  at  5”  . 
an  interior  point  of  9.  As  the  existence  and  properties  of  W  are  more  easily  checked  and 
understood  than  those  of  0"°.  our  next  goal  is  to  derive  conditions  under  which  the  existence  of  0" 
implies  that  W°  exists  and  is  close  to  W*.  We  rewrite  f  from  the  update  law  (2.93)  as 

f(t.i/(l.0))  =  -^»(t.0)[vT(t.0,rX0-r)  +  e(t,D] .  (2.121) 

Using  (2.121)  to  rewrite  f  of  the  averaged  system  (2.94)  as 

7(0)  =  -R(0.rX0-T)  -  avg[0(-.0)e(-.D]  .  (2.122) 

we  state  the  following  result. 

Theorem  2.8:  Suppose  that  0”*.  an  interior  point  of  9.  provides  an  isolated  local  minimum  E((F*)  of 
E(0)  and  that  R((F'  .  W')  is  nonsingular.  Under  these  conditions  there  exist  >  0  and  y„  <  °o  such 
that  if  E(0‘)  <  /i„.  then  the  equilibrium  W°  exists  and 

I0'l’-IT*!  <  yoE(0'’)  •  (2.123) 

Proof:  Factoring  e(t.F’)  as  the  product  of  ej(t).  with  unit  amplitude,  and  /u.  a  scaling  factor. 

e(t.(T)  =  .  avg[ef(- )]  =  1  (2.124) 

we  prove  the  continuity  of  the  function  implicitly  defined  by 

0  =  f '  HQ.fj.)  =  W,)(9—W’)  —  /jl  avg[0(  .9 )e i ( ■ )]  .  (2.125) 

Since  T'(0\(>)  =  0  .  T.XF’.O)  =  R(5’’,0’' ).  and  f  JiW.O)  =  avg[0(-.0^’  )e,(  • )].  the  claim  lollops  Irom 


the  implicit  function  theorem 


S 
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With  the  combination  of  Theorems  2.1  and  2.2  we  have  a  result  which  is  strong  enough  to  be 


practically  applicable  for  the  design  of  adaptive  systems.  Doing  off-line  analysis  or  simulation  of 


(2.1)  for  constant  values  of  parameters,  we  can  check  for  typical  signals  w(t)  entering  the  system. 


whether  or  not  E(0)  has  a  minimum  for  some  value  0"*  in  the  interior  of  the  set  9.  If  this 


minimum  exists  we  can  then  check  the  fact  that  the  slowly  adapting  system  has  an  almost  periodic 


solution  which  preserves  its  u.a.s.  property  in  the  presence  of  a  nonzero  tracking  error.  It  is 


convenient  that  the  restrictions  Theorem  2.2  places  on  the  tracking  error  are  for  the  minimum 


value  of  the  RMS  error  E(F).  However,  this  result  is  local  and  does  not  give  estimates  of  pt0  ,  y0  or 


the  region  of  attraction  of  the  u.a.s.  solution.  Theorems  2.1  and  2.2  are  local  because  their 


hypotheses  depend  on  the  eigenvalues  of  the  constant  matrices  R(0\F  )  and  R(F.F).  respectively. 


Our  next  result  considers  R(0.0"*)  as  a  function  of  0  in  0  and  makes  use  of  the  infinite  time 


approximation  theorem  to  address  these  weaknesses. 


Theorem  2.9:  Suppose  that  0"’  provides  a  local  minimum  E(F)  of  E(0)  in  the  set  Q  0  and 


R(0F)  +  Rt(9.F)  ^  2X1  >0  ¥0  €  BfjSF) 


(2.126) 


If  the  minimum  RMS  error  E(0"*)  satisfies 


E(0‘)  <  \2/(2w„v  1v2  +  Xv  ,) 


(2.127a) 


E(F)  <  0X/(3v) 


(2.127b) 


where 


w,,  =  —  lb,! 

Of,  > 


=  max  HI  d>(  ,0)  HI  $ 


v,  =  max  ill  <£„(  .0 )  ill  $  v, 

I!'  i>^'< 


ith  v  and  \ ,  from  Assumption  2.2.  then 


V.  • 


(0  the  averaged  system  (2.94)  has  a  u.a.s.  equilibrium  0°  such  that 


I  W°-W‘  I  <  -^-E(F)  . 

A 


(ii)  every  solution  of  (2.94)  with  ?F(0)  €  B(0,?T')  satisfies 


(2.128) 


0\t)  €  B(0.0‘)  ¥r  ^  0  .  lim  l5’(r)-9‘°l  =  0 


(2.129) 


(iii)  given  cr  >  0  there  exists  e.(cr)  €  (O.eJ  such  that  for  each  e  €  (O.e.]  the  original  system  system 


(2.93)  possesses  an  almost  periodic  solution  0*(t.€)  which  is  u.a.s.  and 


lim  I0*(t.€)-ST°I  =  0: 
€  —  0 


(2.130) 


(iv)  every  solution  of  (2.93)  with  0(t„.€)  =  (F(0)  €  B(0— c r.W)  satisfies  for  each  e  €  (0.€t] 
0(t.e)  €  B(/3,0‘)  .  1 0(t.€)— ?F(e(t— 10))  I  <  cr  ,  Vt  ^  t„  . 


(2.131) 


lim  I0(t.€)-0,(t.€)l  =0. 


(2.132) 


Proof:  Because  the  Sethna  and  Moran  (1968)  theorem,  together  with  (i)  and  (ii).  implies  (iii)  and 
(iv).  we  need  only  establish  (i)  and  (ii).  Define  the  mapping 


T  :  B(v\-1E(9'’).9'*)  -  B(vA-‘E(D.9'‘)  by 


T(0)  =  W’— R  1(0.9’')avg[0(',0)e(-.^’)]  . 


(2.133) 


The  inequalities  (2.127)  are  sufficient  for  T  to  be  a  contraction  mapping  on  B(v  A~1E(S’’).ir);  hence. 
T  has  a  unique  fixed  point  W"  €  B(  v  A_1H(9"’).0"’).  Clearly,  the  fixed  point  of  T  is  a  solution  of 
f  (0)  =  0  Thus  (2.94)  has  an  equilibrium  S’"  satisfying  the  bound  (2.128).  We  establish  that  this 

equilibrium  is  exponentially  stable  with  the  Lyapunov  function  Y(0)  =  -i-  0‘0.  Rewriting  (2  94)  in 

the  I orm 


(2.134) 


—  F  =  -R(0\D(0"-0"°)  -  [b(0")-b(0"0)] 
dr 

+  (R(9\?T)  -  R(0"o.r)](0""-D  . 

where 

b(0)  =  avg[<£(  ,0)e(  ,?F’)]  . 

it  follows  that 

—  V($-W°)  ^  2[— X-K2w0v  jV^X-1  +  7,)E(0')]V(g'-g'0)  (2.135) 

dr 

for  all  W  €  B(f3.^*).  The  inequalities  (2.127b)  and  (2.135)  imply  that  W°  is  exponentially  stable 
and  that  its  region  of  attraction  includes  B(20/3.?T'’).  that  is.  there  exists  Xj  >  0  such  that  solutions 
of  (2  94 )  with  0(7,,)  €  B(20/3  .  W")  Q  B(j 3.0"')  for  any  r„  ^  0  satisfy  for  all  7  ^  7,, 

I  0"( 7 ) — W"  I  ^  I  W(t„)— I  exp[— X,(7— 7,,))  .  (2.136) 

To  show  that  the  region  of  attraction  of  W"  includes  B(0.0"‘).  that  is.  to  establish  (2.129)  it  is 
enough  to  show  that  solutions  of  (2.94)  with  F(0)  €  B(0#*)  enter  B( 2/3/3 .^  ’)  in  finite  time.  Using 
the  form  of  f  in  (2.122)  we  compute  the  inequality 

—  VrtT-D  <  -XI  IMP  I  2  +  7E(0r’)l9‘-r  I  .  (2.137) 

d7 

Choosing  8  >  0  such  that  S  +  vX'!E(?')  <  0/3.  it  is  clear  from  (2.137)  that  solutions  of  (2.94) 
beginning  in  B(0.£T’)  enter  B(S  +  v  X~'E(0"’).0"')  Q  B(20/3.0"1’)  in  finite  time. 

□ 

I  he  stability  condition  (2.126)  is  again  the  signal-dependent  SPR  condition.  However,  it  is 
nut  evaluated  pointwise  in  0  for  each  0  in  a  ball  around  0"’  rather  than  at  only  the  point  0  =  0. 
The  formula  (2  106)  lor  R(0.0")  is  still  valid:  hence  the  lrequency  domain  interpretations  ol 
!2  126)  are  analogous  to  those  of  (2.109).  finally,  we  point  out  that  as  0  varies  over  B(0.0"’)  the 
transfer  lunction  W,  [((TVs)  does  not  change.  That  is  the  condition  (2.126)  is  a  signal-dependent 
SPR  condition  on  the  fixed  transfer  function  Wtl(0’.s)  lor  the  different  signals  d>(t.0)  as  0  varies 
o  .er  t he  bal  1  B(  0.0"’ ). 


2.7 .  Concluding  Remarks 

Pursuing  the  intuitively  appealing  distinction  between  states  and  parameters,  we  have 
established  conditions  for  the  existence  of  an  integral  manifold  —  the  slow  manifold  —  and  used  it 
for  an  exact  description  of  the  slow  adaptation  process.  Conditions  for  the  exponential  attractivity 
of  the  slow  manifold  are  formulated,  and  the  exponential  attractivity  is  shown  to  imply  that  the 
stability  properties  of  a  solution  of  the  reduced-order  system  on  the  manifold  are  also  the  stability 
properties  of  the  corresponding  solution  of  the  full-order  system.  Based  on  this  reduced-order 
exact  description,  we  have  examined  the  validity  of  earlier  results  obtained  via  the  averaging  of 
what  is  now  shown  to  be  a  "frozen  parameter"  approximation  of  the  slow  manifold.  A  particular 
model  reference  adaptive  control  system  is  shown  to  possess  an  exponentially  attractive  slow 
manifold.  The  stability  of  this  system  is  then  analyzed  via  averaging  of  the  equation  describing  the 
motion  in  the  manifold.  This  analysis  extends  and  completes  earlier  results  based  on  a  linearization 
near  a  "tuned"  svstem. 


CHAPTER  3 


INTEGRAL  MANIFOLDS  OF  SLOW  ADAPTATION  IN  DISCRETE  TIME 


3.L  Introduction 

Many  discrete  time  adaptive  control  schemes  are  described  by  ordinary  difference  equations  of 
the  form 

x(k+l)  =  A(0(k))x(k)  +  B(0(k))w(k)  .  (3.1) 

0(k+l)  =  0(k)  +  €f(k,0(k).x(k))  .  (3.2) 

where,  as  in  (2.l)-(2.2).  x  contains  the  states  of  the  plant,  controller,  filters,  etc.,  and  0  is  the  vector 
of  adjustable  parameters.  We  remark  that  if  the  update  law  is  of  the  Newton  or  least-squares 
type,  then  0  contains  the  columns  of  the  Newton  matrix.  As  in  Chapter  2.  we  first  derive 
conditions  under  which  (3.1 )-( 3.2 )  possesses  an  integral  manifold  Me  of  the  form 

Me  =  (k.e.x  :  x  =  g(k,0;e)|  .  (3.3) 

and  then  consider  its  attractivity.  Restricted  to  the  slow  manifold  Me  the  system  (3.1  )-(3.2) 
evolves  according  to  x(k)  =  g(k.0(k):e)  and 

0(k+l)  =  0(k)  +  €f(k,0(k).g(k.0(k):e))  .  (3.4) 

which  is  in  the  standard  form  for  averaging.  Discrete  time  averaging  theory  relates  the  solutions  of 
(3.4)  to  the  solutions  of  the  ordinary  differential  equation  (ODE) 


±3  =  7(5) 

dr 

where 


f(0)  =  lim  —  7*  f(j.0.g(j.0:O))  (3.6) 

Leaving  ihe  application  of  this  theory  to  a  specific  adaptive  system  for  Chapter  4.  we  give  proofs  of 
several  discrete-time  averaging  theorems  using  deterministic  assumptions.  By  making  appropriate 
assumptions  on  the  stochastic  process  which  generates  the  inputs  to  (3.1 )-( 3.2).  we  conclude  this 
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chapter  with  a  lemma  that  relates  the  trajectories  of  some  stochastic  adaptive  control  systems  in 
the  form  (3.1)-(3.2)  to  solutions  of  the  ODE  (3.5). 

3.2.  Approximation  of  the  Slow  Manifold 

-As  in  the  continuous-time  case,  an  integral  manifold  M(  of  (3.l)-(3.2)  is  defined  by  the 
statement  that  if  x,0  is  in  at  k=k0,  then  it  is  in  for  all  k^k,).  that  is. 

(ko.0(ko).x(ko))€Mt  =>  (k.0(k).x(kM€M«  Vk  Vk0€Z  .  (3.7) 

In  general,  solving  for  the  function  g(k.0;£).  which  determines  M{  via  (3.3).  is  as  difficult  as  solving 
the  complete  system  (3.1)-(3.2).  Our  approach,  then,  is  to  find  an  easily  computable  and 
meaningful  approximation  of  g(k.0:e).  Mote  that  at  e=0.  0  is  constant  and  (3.1)  is  a  linear  time- 
invariant  system  w ith  input  w(k).  Hence,  we  can  compute  the  solution  of  (3.1  )-(3.2)  for  6=0.  The 
variation  of  constants  formula  applied  to  (3.1)  with  0(k)  =  0.  a  constant,  gives 

x(k)  =  Ak"it°(0)x(ko)  +  Z  Ak-,"'(0)B(0)w(i)  .  k£k„+l  .  (3.8) 

.=k0 

Assumption  3.1:  There  exist  a  set  0  and  constants  K€[l.oo)  and  A0€(0.1)  such  that 

|A‘(0)|  <  KAd  .  i^O  .  ¥060  .  (3.9) 

□ 

Making  use  of  this  stability  assumption,  we  let  k„— oo  in  (3.8)  and  take  the  steady-state  response 

*'<k.0)  =  Z  Ak-,_l(0)B(0)w(i)  (3.io) 

l=— ‘-o 

as  the  manifold  function  g(k.0:O)  =  iHk.0)  defining  the  frozen  parameter  manifold  M„.  This 
(unction  is  both  meaninglul  and  easy  to  compute.  Therefore,  in  addition  to  proving  that  Me  exists, 
we  adopt  the  goal  of  showing  that  g(k,0:O)  =  y(k.0)  is  a  good  approximation  of  g(k.9:e)  for  small  €. 
In  order  to  meet  both  goals  simultaneously,  we  introduce  the  deviation  of  x  from  t/(k.0)  as  a  new 


state  variable 
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i 
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z  =  x  —  v(k.0)  .  (3.11) 

and  transform  (3.l)-(3.2)  into 

z(k+l)  =  A(0(k))z(k)  —  G(k+1.0(k).z(k))  (3.12) 

0(k+l)  =  0(k)  +  €F(k.0(k).z(k))  (3.13) 

where 

G(k+1.0,z)  =  Kk+1.0+€F(k.0.z))  —  v(k+1.0)  (3.14) 

F(k.0.z)  4  f(k.0.i/(k.0)  +  z)  .  (3.15) 

The  goals  are  met  by  proving  that  (3.12 )-(3. 13)  possesses  an  integral  manifold  Mf  determined  by 

Me  =  |k,0,z  :  z  =  h(k.0:e)}  (3.16) 

with  h(k.0:e)  =  0(e). 


Before  proving  the  existence  of  h(k.0;€).  we  consider  formal  (without  proof)  approximations 
of  h(k.0;€).  From  the  definition  of  it  follows  that  the  function  h(k.0;e)  evaluated  along  a 
trajectory  of  (3.12M3.13)  which  is  in  Mt  must  satisfy  (3.12M3.13)  with  z  replaced  by  h- 
Performing  this  substitution,  we  get  the  functional  difference  equation 

h(k+1.0+€F(k.0.h(k.0;e));e)  =  A(0)h(k.0;e)  -  G(k+1.0.h(k.0;e))  .  (3.17) 


v, 

V, 


.V 

■  b 


This  is  no  longer  an  ordinary  difference  equation  in  k  because  of  the  variations  in  the  second 
argument  of  h.  Rewriting  (3.1 7)  as 

h(k+1.0:e)  =  A(0)h(k.0:«)  -  G(k  +  1 ,0.h( k.0:e))  , 

(3.18) 

-  [h(k+l .0+eF(k.0,h(k.0:e)):e)  -  h(k  +  1.0;e)] 

we  bring  (3.17)  to  a  discrete-time  analog  of  the  partial  differential  equation  (2.23).  Under  an 
appropriate  smoothness  assumption  we  can  approximate  h(  k,0:e)  by  a  power  series  in  e 


h(k.0:e)  =  h,,(  k  .0 )  +  €h,(k.0)  +  6‘h:(k.0)  +  ■  •  • 


(3.19) 


i 


Substituting  this  series  for  h(k.0;e)  in  (3  18)  and  equating  the  coefficients  of  like  powers  of  €.  the 
equation  for  h,,(k.0)  is  an  ordinary  difference  equation  in  k  whose  steady-state  response  is  zero: 
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ho(k+1.0)  =  A(0)ho(k.0)  =>  ho(k.0)=O  . 


For  the  e  term  h,(k.0)  the  ordinary  difference  equation  is 


(3.20) 


h,(k+1.0)  =  A(0)h,(k.0)-i/e(k+1.0)F(k.0.O) 


(3.21) 


and  its  steady-state  solution  for  each  fixed  0  is  given  by 


ht(k.0)  =  Z  Ak-1-‘(0M i+ 1  )F( i.0.0)  . 


(3.22) 


Here  use  is  made  of  F(k,0.ho(k.0))  =  F(k.0,O).  This  process  of  successively  evaluating  the  hj's 
continues  with  each  coefficient  hj(k.0)  in  the  expansion  (3.19)  being  the  steady-state  response  of  a 
linear  ordinary  difference  equation  in  k.  parametrically  dependent  on  0.  and  driven  by  terms  with 
h,  and  derivatives  of  h,  only  for  i  <  j. 


3.3.  Existence  of  a  Slow  Manifold 

As  in  the  continuous-time  case,  we  derive  conditions  under  which  Mf  exists  by  constructing  a 
map  T4  whose  fixed  point  is  h(k.0:e)  and  finding  conditions  for  Te  to  be  a  contraction.  We  first 
specify  a  closed  subset  of  a  Banach  space  in  which  to  search  for  h(k,0;€).  Letting  the  space  be  the 
set  of  continuous  functions  H(k.0)  equipped  with  the  norm  ||H||  =  sup  |H(k,0)|.  we  use  positive 

ZxRn,, 

constants  D.  A  to  define  our  closed  subset  H(D,  A)  as 

H(D.A)  =  (H:ZxRne-*Rn’'  ||Hj|^D  and  |H(k.0)-H(k.0)|  A  |0— 0|¥k6Z:V0.06R‘^}  .(3.23) 

Recall  that  our  goal  is  not  only  to  prove  that  h(k.0:e)  exists  but  also  to  show  that  D  and  A  are 
( )<  € ).  In  addition  to  the  stability  assumption  we  require  the  parameterization  A(0).  B(0)  to  be 
continuously  differentiable  and  to  have  Lipschitz  first  derivatives  for  0€0  and  w(k)  to  be 
uniformly  bounded  We  quantify  this  requirement  in  the  following  assumption. 


Ass’'  ,  tion  3.2:  There  exist  positive  constants  v.V].v2  such  that 


|^(k.0)|  ^  v.  |v4k.0)|  $  v,  .  ji/r/(k.0) — izcCk.O))  ^  v2|0— 0| 


(3.24) 
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for  all  k€Z,  and  all  0,060. 


□ 


With  this  assumption,  the  existence  of  h€H(D.A)  implies  that  g  is  uniformly  bounded  and 
Lipschitz  continuous  in  0.  Finally,  we  must  use  a  parameter  update  law  (3.2)  with  F(k,0.x)  being 
bounded  and  Lipschitzian  in  0,x  uniformly  with  respect  to  k6Z,0G0.  and  x  in  compact  sets. 


Assumption  3.3:  There  exist  nondecreasing  positive  functions  pF(D).  p@(D),  and  pz(D)  such  that 


|F(k,0,z)|  <  Pf(D).  |F(k.e,z)-F(k.0.z)l  ^  PeCD^t0— 01  . 
|F(k.0.z)— F(k.0.z)|  <  pz(D)|z— z| 

for  all  kGZ.  for  0.069.  and  for  all  z.z  with  |z|  ^  D.jzj  $  D. 


(3.25) 


□ 


Anticipating  the  same  type  of  stability  conditions  that  were  encountered  in  Chapter  2,  we 
again  introduce  a  modified  system  where  6  is  replaced  by  p(0)€9  in  A.  G.  and  F.  Notice  that 
G(k  +  1.0.z)  contains  v(k+l .0+€F(k.0.z)).  In  order  to  ensure  that  p(0)+eF(k.p(0).z)60  we  must 
choose  p(0)  strictly  inside  0.  Let  0,(6. D)  be  a  compact,  convex  subset  of  0  such  that  069,(6.0) 
implies  B(epF(D).0)£0.  We  take  p(0)  to  be  the  unique  element  of  0,(e.D)  which  is  closest  to  0. 
namely. 

p(0)  =  arg  min  |p-0|  .  (3.26) 

pfc©  (t.D) 

Remark  3.1:  With  this  definition  p  is  potentially  a  function  of  €  and  D.  However,  the  meaning  of 
p  and  the  choice  of  the  set  9,  are  generally  clear.  Hence,  for  notational  simplicity  we  do  not 
explicitly  indicate  the  dependence  of  p  on  €  or  D. 

□ 

We  henceforth  analyze  the  modified  system 

z(k+1)  =  A(  p(0(  k ) ) )/.( k )— G(k  +  1  ,p(  0(  k  )).z(  k ))  (3.27) 

0(k  +  l  )  =  0(k)+eF(k.p(0(k)).z(k))  .  (3.2K) 

This  modified  system  is  similar  to  but  not  identical  to  the  usual  kind  of  parameter  update  laws 
incorporating  projection,  which  have  the  form 


‘ 0  vSr 


jJi  ■ciTia.C.  v-  lj  £m ~  » a  «-i-» 
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0(k+l)  =  p[0(k)+€F(k.0(k).z(k))]  . 


(3.29) 


Update  laws  such  as  (3.29)  ensure  0(k)€0  for  all  k  which  makes  the  p(0)  in  A  and  G  unnecessary, 
whereas  (3.28)  allows  0(k)  to  leave  0  but  0(k+l)— 0(k)  is  always  computed  using  p(0(k))€0.  The 
study  of  (3.29)  introduces  technical  difficulties  at  the  boundary  of  0-  A  topic  of  future  research  is 
how  to  avoid  the  technical  difficulties  associated  with  the  usual  projection  algorithm  (3.29)  while 
avoiding  the  potential  unboundedness  associated  with  the  modified  update  law  (3.28). 


With  the  use  of  A(p(0)),  B(p(0)).  G(p (0)).  F(k.p(0).z).  the  fact  that  |p(0)-p(0)|  ^  |0~0| 


implies  that  the  modified  system  satisfies  Assumptions  3. 1-3.3  for  all  OCR"6.  We  describe  the 


dependence  of  F(k.p(0).H(k,0)  and  G(k+l,0,H(k.0))  on  0  for  HGH(D.A)  by  defining  pj(D.A)  and 


p2(D.A)  such  that 


|F(k.p(0).H(k.0))  -  F(k,p(0))H(k.0))|  ^  p,(D.A)|0-0| 
|G(k+l.p(0).H(k.0))  -  G(k+l.p(0).H(k.0))|  <  ep2(D.A)|0-0| 


(3.30) 


for  all  kGZ.  for  all  0.0CR"6  and  all  H€H(D,A).  It  follows  from  Assumptions  3.2  and  3.3  that  p, 


and  p2  exist  and  can  be  chosen  to  satisfy 


p,(D.A)  <  p9(D)+Ap2(D).  p2(D.A)  v1p,(D.A)+v2pF(D)  .  (3.31) 

Our  first  step  in  constructing  the  map  T€  is  to  define  0j(i:k,0,6)  for  i^k  as  the  solution  of  the 


end-value  problem 


0/i+l)  =  0J(i)+eF(i.p(0J(i)).H/i.0/i))).  0/k)  =  0  . 


(3.32) 


where  the  subscript  j  implies  dependence  on  Hr  We  bound  the  dependence  of  0j(i;k.0,€)  on  0  and  Hj 


in  the  following  lemma. 


Lemma  3.1:  Suppose  that  Assumptions  3. 1-3. 3  hold.  Lei  €j(D.A)  —  1/P](D.A).  For  each 


e€[0.6| ).  if  H.eH(D.A)  and  H,n€H( D.A).  then 


|0.(i:k.0.e)  —  0,( i:k,0.e)|  ^ 


l-ep,(D.A) 


Ml  ■ 


(3.33) 


v  <  » J,  *  »>Ji  fj 
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Jjtiik.fl.e)— &m(i:fc.O.€)|  « 


1— ep/D.A) 


k-i 


-1 


!|HrHm|| 


(3.34) 


for  all  i  ^  k 


Proof:  Letting  0/0  and  0/0  denote  0j(i:k.0.€)  and  0/i;k.0.€).  respectively,  we  have 


|0/.+l)-0/i+l)|  £  |0j(i)— 0(i)|— €Pi(D.A)|0j(i)— 0j(i)| 

=  [  1  — €Pi(  D.A  )](0j(i) — 0/ i )( 

for  all  k— 1.  Dividing  both  sides  of  (3.35)  by  1— €P](D.A)>0,  we  get  the  desired  result. 


(3.35) 


10/0-0/01 


1— ep/D.A) 


|0/i+l)-0/i+l)| 


1 


1-ep/D.A) 

1 


k — i 


|0/k)— 0/k)| 


1—  €Pi(D.A) 


k— i 


(3.36) 


Ml 


Letting  0/i)  and  0m(i)  denote  0/i;k.0.€)  and  0m(i;k,0.€).  respectively,  we  have 


|0/,+  l)-0m(.+l)|  >  |0/i)-0in(i)|-6p,(D.A)|0J(O-0in(i)| 

—  €pz(  D)|H/  i.0m(  i )  )—)!„,(  i  .0m(  0  )| 


(3.37) 


for  all  i^k— 1.  We  again  divide  by  1— ep^D.A)  to  obtain  the  desired  result. 


|0/i)-0m(i)|  < 


1 


1  — €P)(  D.  A ) 


ie,(i+i)-e„(,+i)|  +  IIh,-hJ 


k — i — 1 


1-€P,(D.A) 


€^(D)  m,_u  ,|  1-€Pi(d-A) 
Hml1  'ep/D.Al- 


1— ep/D.A) 


1— €P|(D.A) 


k— i 


-1 


(3.38) 


□ 


The  second  step  in  the  construction  of  the  map  T,  requires  the  stability  of  the  iinear  time- 
.  arving  system 

z(,+ 1)  =  Afpfe/OOzti)  .  (3.39) 
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r(k)  ^  K.r(k0)  II  [X+Kp(0] 

>=*o 


(3  46) 


Proof:  Letting  rj(k)  =  X  *r(k).  we  have 


rj(k)  <  Kr,(k0)  +  £  -^-p(i)r,(i) 


(3.47) 


Letting  r2(k0)  =  Krj(k0)  and 


r2(k)  =  Kr2(k0)  +  ^  J^-p(i)r2(i) 


(3.48) 


we  see  that  r2(k)  satisfies  the  scalar  ordinary  difference  equation 


r2(k+l)  =  [l  +  -£-p(k)]r2(k) 

A 

=  I  n  (l  +  ^p(i)]|r2(k0) 


(3.49) 


Comparing  (3.47)  and  (3. 48)  it  is  clear  that  r^k^r^k):  hence. 


r(k)  =  X^fk)  ^  r?(k<,)Xk  II  [l-^-p(i)] 

i=kn  X 

=  Kr(kJXk~k°  Vfl+JipG)] 
>=k0  X 

=  Kr(k0) kn[X+Kp(i)]  . 

i=k„ 


(3.50) 


Proof  of  Lemma  3.2:  Letting  A(i)  denote  A(p(0,(i))).  it  follows  from  Assumption  3.3  and  (3.41) 


j.A(nj) — A(n-»)j  ^  €app(D)|n]  n2| 


(3.51) 


In  order  to  prove  (3.43)  it  is  sufficient  to  show  that  the  solution  of  (3.39)  satisfies 


|z(n))|  ^  Kjz(n2)|Xi''  I>2(e.D)  .  n^n? 


(3.52) 


lor  an  arbitrary  z(n2)  and  arbitrary  n2.  Lor  any  integer  u.  we  can  rewrite  (3.39)  as 


z(i+l)  =  A(u)z(i)  +  ( A(i) —  A(u))z(i) 


Applying  the  variation  of  constants  formula  to  (3.53)  and  taking  norms  we  have 


|z(nj)|  ^  K|z(n2)|X0ni  +  £  KA0n‘  1  '€apF(D)|i— u|  |z(i)|  . 


which  in  light  of  Lemma  3.3  implies 


|z(n!)|  ^  K|z(n2)|  II  (X0+eKapF(D)|i— u|)  . 

i=n2 

r:  ni“Il2  rii“’n2“—  1 

ror  any  given  nl  and  n2  we  choose  u  as  the  integer  — - —  or  - - - 

^  (n j— n2)2  x2 

2.  I'~ ul  ^  - r— — ■  Noting  that  ln(x]+x2)  <  ln^H —  we  have 


h"1  n,-l 

II  [X„+€K.apF(D)|i— u|]  =  exp  In  II  [X0+€KapF(D)ji— u|] 

*■  ‘2  >=n2 


=  exp  ln[X0+€KapF(D)|i-u|] 


<  exp  Y.  ln[X0+€KapF(D)^i-^i]  +  £  eKapF(D)[|i-u|-^--^ 

i=n2  ^  i=n2  ^ 


^  exP  X  ln[X„+€KapF(D)— !— — l] 

i=rv2  ** 


=  [X„+eKapF(D)^ir-^ 

4 


Thus.  (3.55)  is  replaced  by 


|z(n[)|  <  K|z( n2 )|[X,i+€KapF(D)- 


For  n,— n-> ^ \(e.D).  we  have 


A„+€KapF(D)--'  ,n :  ^  X,(e.D)  : 
4 


hence.  (3.43)  holds  for  n,—  n2 ^ N( e.D).  At  n,=n2  +  \(e.D)  we  have 
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X,N(tD)(€.D)  =  K[\0+€KapF(D)^i^l]N(<D>  . 


(3.59) 


which  implies  that  for  any  integer  m^O 


|z(n2+mN(€.D))|  <  XimN(t  D)(€.D)|z(n2)| 


(3.60) 


Since  any  interval  of  length  n!— n2  can  be  broken  up  into  an  interval  of  length  mN(e.D)  and  one  of 

length  less  than  .\(e,D).  this  completes  the  proof.  Note  that  the  proof  holds  for  any  integer  N  in 

place  of  N(e.D).  The  particular  choice  of  N(e.D)  given  by  (3.42)  approximately  minimizes  Xj(€,D). 

□ 

Remark  3.2:  N(e.D)  =  0  is  a  degenerate  case  where  (K— l)X0<€-  ^ — -.  When  K  is  so  close  to  1 

or  X0  is  so  close  to  zero,  we  let  X0=KX0  and  note  from  (3.9)  that  |A(0)|  ^  X„  V(0)€0.  Hence, 
solutions  of  (3.39)  satisfy  |z(i)|  ^  (X0)'  l°|z( i0)|  no  matter  how  fast  or  slowly  0j  moves. 


From  the  fact  that  K(1/N,<  1  +  — — — .  and  from  equations  (3.42).  and  (3.44),  we  bound  X,(e.D) 

i\ 


X,(€.D)  ^  1  +  - 


eapF(D)K(K— 1 ) 

eapF(D)K 
1_  4X„(k-l) 


eapF(D)k(k-l) 

1  +  - — -  X0 

4Xo 


(3.61) 


Since  X,,  <  1  it  is  clear  from  (3.61 )  that  €  can  be  chosen  small  enough  so  that  Xj(e.D)<  1 :  hence,  for  € 
sufficiently  small  (3.39)  is  exponentially  stable.  We  complete  the  construction  of  the  map  T(  with 
the  pointwise  definition 


(TfHj)(k.0)  =  £  <l»J(k-l.i:k.0.£)r»(i+i.p(0J(i:k.0.e)).H;(i.0J(i:k.0.€)))  . 


(3.62) 


Because  (i(k.p(0 ).HJ(  i.0 ) )  is  uniformly  bounded  and  <t>  is  exponentially  decaying  tor  e  sufficiently 
small  the  right-hand  side  ol  (3.62)  is  bounded  for  each  k€Z  and  each  0€R'".  If  h  is  a  fixed  point 
ol  T„.  then  given  any  k, ,.  0, ,  choosing  z(k,,)=  h(k,,.0,,;e)  results  in  a  solution  z(k).  0(k)  such  that 
/( k  1  =  h(k.0(k);e).  Hence,  the  fixed  point  of  T.  is  indeed  the  manifold  function  h  which  we  are 
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seeking.  We  derive  conditions  for  T*  to  be  a  contraction  on  H(D.A)  in  the  following  lemma. 

Lemma  3.4:  Suppose  that  Assumptions  3. 1  -3.3  hold  and  that  (3.41)  holds.  If  €>0.  D>0,  and 


A>0  satisfy 


1-X,(«.D) 


AjCe.D)  <  1— epi(D.A) 


P2(D‘a) 


KavjPpCD) 


6  t™t(Kv‘+4)  <  1  • 


(3.63) 


(3.64) 


(3.65) 


(3.66) 


where  A/e.D)  is  given  by  (3.44).  then  T€  is  a  contraction  mapping  on  H(D.A). 


Proof:  Let  HrHnl€H(D.A)  be  arbitrary.  The  first  bound  (3.63)  is  the  easiest  to  obtain.  As 
ev,pF(D)  bounds  |G(H-l.p(0,(i:k.0.€)).HJ(i.0J(i:k.0.e)))|.  we  have 


k“i  KviPc(D) 

|(T<H,)(k.0)|  <  ^KX,i-|-'(€.D)6v,pf(D)  =  6  -D) 


(3.67) 


hence.  (3.63)  ensures  ||T(Hj|  ^  D.  The  bound  (3.64)  arises  in  the  derivation  of  (3.65)  and  (3.66) 
which  guarantee  that  ((T^Xk.#)— (TtHj)(k.0)|  ^  A|0— 0|  and  ||TfHj— T{Hm||  <  ||Hj — Hm||. 
respectively.  The  most  difficult  step  in  establishing  (3.65)  and  (3.66)  is  determining  the  dependence 
of  <J>,(n1.n2;k.0.€)  on  0  and  Hr  We  do  this  now.  Denoting  <t>J(ni.n2;k.0.€).  <J>m(n) .n2:k.0.e), 
0  (i:k.0.e),  and  0m(i:k.0.e)  bv  <J>(n1.n2).<&(n,.n2).  0(0.  and  0(0.  respectively,  we  write  the  ordinary 
difference  equation  in  n. 


<t>fn,  +  l  ,n:)— <t»( n ! -I- 1  ,n?)  =  A( p(0( nj )))[<J>(n( ,n: )— <t>( n] ,n; )] 

+  [A(  p(0(  n, )))— A(  p( n ! ) ) )]<!>(  n  i  .n2 ) 

Applvins  the  variation  ot  constants  formula,  we  obtain 


(3.68 ) 


[4>'  n  i  .n:  )— ■ <J>(  nf  ,n2  )]  =  ['Ll  n:.n2  )—<!>(  n2.n2  )J  +  £  4>(  nj  —  1 .  i  )[  A(p(0(  i  1 ) )— A(  p(  0(  i  0  )]<t>(  i.n2)  (3.60  ) 


W:  vrv« 


i 


59 


for  all  n2^nt^k.  Using  <t>(n2.n2)  =  4>(n2.n2)  =  I,  (3.41),  (3.43)  and  the  triangle  inequality,  we 
bound  <J>(n2.n2)~ <J>(n!.n2)  by 

nj— l 

|4>J(n,.n2;k.0.€)-<I>rn(n1.n2;k.d.€)|  ^  K2a\r,-1_nj(€.D)  £  |0jG:k.0.€)-0m(i;k.0.e)|  (3.70) 


Note  that  (3.64)  implies  e<e1(D.A);  hence,  the  bounds  (3.33),  (3.34)  hold.  Substituting  from 
Lemma  3. 1  we  arrive  at  the  kev  to  (3.65). 


|<J>j(k— l.n:k. 0  .«)—<!>  j(k—l  .n;k,0,e)| 


<  K2aXlk-2-n(€.D)  |0— 0|  £ 


=  K2a|0— ^^^^“"(e.D) 


k2a|0— 0|  X,(€.D) 


-\,K-I_n(€-D) 


(3.71) 


and  the  key  to  (3  66). 


|<t>,(  k-1  ,n;k. 0.6 )-<!>„,( k-1  .n;k,0.e)| 

^  K2aAr,-2(€J3)||HrHm||-^Lz: 


k2ap,(  D)||H,— Hm||  X,(«.D) 
€pf(D.A)X,(€.D)  l-ep,(D.A) 


(3.72) 


-X,‘-,-n(€.D)  -  €Pi( D. A )( k—  1  — n )X  *t-i-,'( g,D) 


Using  the  inequality  (3.71 )  we  bound  the  dependence  of  T€Hj  on  0.  With  the  same  notation  as  in 
( 3  66 )  and  j=m.  the  triangle  inequality  gives 
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|(TtHj)(k.0)-(TeHJ)(k.0)| 


£  |<X>( k — 1  ,i)[G(i+  l.p(0(i)).Hj(i,0(i)))—  G(i+l,p(0(i)),Hj(i.0(i)))]| 

1  =  — o o 

+  £  |[<J>(k— l,i)— <t>(k— l.i)]G(i+l.p(0(i)).Hj(i.0(i)))| 


<  £  ekAf-'-He.D^CD.A) 


(3.73) 


+  'f'  «VP,<P>K;»M|  M«-D>  m 

£P,(D.4H,(«.D)  1-€p,(D.A)  1 


$  €  T-€p7iXa)^x7(^)  P2(d-a)  +  [l-X^e.D)j  10-01  • 

where  the  last  line  holds  because  of  (3.64).  Thus.  (3.63)-(3.65)  imply  that  T^HjfHKD.A)  if 
Hj€H(D.A).  Denoting  <!>_,( k —  l.i:k.0.€).  0j(i;k.0.€)  by  <C>j(k-l ,i).  0j( i),  respectively,  and  similarly  for 
we  have 

|(TsH,)(k.0)-(T<Hm)(k.0)| 

^  £  |<!>i(k-l.i)[G(i+l.p(0J(i)).HJ(i.0j(i)))-G(i+l.p(0m(i)).Hj(i.0m(i)))]| 

l  =  — OL> 

+  £  |4>,(k— 1  ,i)[G(i+l ,p(0m(i)).HJ(i.0(i)))—  G(i+l,p(0m(i)).Hm(i.0m(i)))]| 

+  £  |[<3>j( k —  1  ,i ) — $m(k—  l.i)]G(i+l.p(9m(i)).Hm(i,0m(i)))| 

\  =  —oo 


1— €P](IXA) 


+  £  ekvlP2( D)||H  — Hm|(X,k-l_l(€.D) 


L  V  6v ,pF( D)k2ap/ D)||Hj— H„ 
epf(D.A)\,(e.D) 


\,(€.D) 


-xr'-'fe.D) 


(3.74) 


-  €P|(D.A)(k— 1— i)X,k_l-1(€.D) 


_  ep.imllH.-Hj |v  ^  eK  v.kap,(D) 

1-\,(€.D)  V|  1— €p,(  D.A)— \,(e.D)  P:  '  [1— \,(€.D)J 

€p.i  1)  1 

$  — — . — -(kv,+A)||n-HJ|  . 

1  — A  ,i  e.D)  1  11 


m 


m&sm 
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Remark  3.3:  Lemma  3.4  uses  X^e.D)  given  by  (3.44)  only  to  imply  that  the  stability  bound 
(3.43)  holds.  If  (3.43)  can  be  established  with  X)  replaced  by  X/.  then  Xj'  can  be  used  in  (3.63)- 
(3.66).  For  example,  if  there  exists  a  positive  definite  matrix  F  such  that 

At(0)PA(0)  <  X()P  ¥060  .  (3.75) 

then  (3.43)  is  satisfied  with  Xi(€,D)  =  X0  and  K  =  (XmaxP/XminP)V:. 

□ 

In  light  of  (3.61).  the  fact  that  X0<1.  and  the  fact  that  Xi(e.D)  is  a  nondecreasing  function  of 
e  and  D.  it  is  clear  that  for  any  fixed  D0>0.  Ao>0  we  can  find  an  e2(D0.A0)  such  that  (3.63M3.66) 
hold  for  all  €6[0.€2).  Because  pF,  p2,  Pi.  and  p2  are  nondecreasing  functions  of  D.  A,  it  follows  that 
(3.63)-(3.66)  hold  with  D  =  (e/e2)D0  and  A  =  (e/€2)A„  for  all  e€[0.e2).  We  illustrate  the  last 
statement  with  (3.63), 
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Remark  3.4:  Up  to  this  point  the  only  assumption  we  have  made  about  the  external  inputs  to  the 
system.  w(k)  and  f(k.v).  is  that  they  are  uniformly  bounded.  If  they  are  periodic  (almost 
periodic),  then  ^(k.6)  and  h(k.0;e)  are  periodic  (almost  periodic)  in  k. 

□ 

Before  considering  the  attractivity  of  Mt  in  the  next  section,  we  give  an  instability  result. 
The  evolution  of  (3.27)-(3.28)  restricted  to  M<  is  governed  by  the  reduced-order  system 

0(k+l)  =  0(k)  +  eF(k.p(0(k)).h(k.0(k):«))  .  (3.77) 

The  proof  of  the  following  theorem  is  identical  to  the  proof  of  Theorem  2.2. 

Theorem  3.2:  Under  the  conditions  of  Theorem  3.1.  suppose  that  €<€2(D0.A0).  Let  0  (k)  be  a 
solution  of  (3.77).  If  0*(k)  is  an  unstable  solution  of  (3.77).  then  z  =  h(k.0  (k)'.e).  0  (k)  is  an 
unstable  solution  of  (3.27)-(3.28). 

□ 

3.4.  Attractivity  of  the  Slow  Manifold 

Theorem  3.2  showed  that  the  existence  of  VI  e  is  sufficient  to  prove  that  instability  in  the 
reduced-order  system  (3.77)  implies  instability  in  the  full-order  system  (3.27)-(3.28).  However, 
existence  of  \1*  is  not  sufficient  to  show  that  the  existence  of  a  stable  solution  of  (3.77)  implies  the 
existence  of  a  stable  solution  of  (3.27 )-( 3.28).  In  this  section  we  establish  the  exponential 
attractivity  of  M(  which  is  sufficient  to  show  that  the  existence  of  a  uniformly  (asymptotically) 
stable  solution  of  (3.77)  implies  the  existence  of  a  uniformly  (asymptotically)  stable  solution  of 
( 3.27 )-( 3.28 ). 

We  begin  with  a  boundedness  result  tor  z(k)  which  justifies  Remark  2.8. 

Lemma  3-5:  Suppose  that  Assumptions  3. 1-3.3  hold  and  that  (3.41)  holds,  (liven  any  fixed 
A  €  (A.,.1).  1),,  >  0.  and  D,  ^  D,,.  let  p(0)  lake  salues  in  01(6.1))).  There  exists  €j(D,,.I)| .A )  >  0 
such  that  t or  each  €€[().€,]  if  lz(k,,)l  ^1),/K  and  0(k,,)l  <  oo.  then 


SL 


I  z(k)  I  <s  Kr  10  lz(k0)l  +  (e/€3)D0(l— A  °)  ^  Di 


(3.78) 


Proof:  Choose  €3(D0,D1.A1)  so  that 


AjUj.D,)  ^  Aj  .  €3KvjPp(Dj)  ^  Dq(  1— AiCe^.Di)) . 


(3.79) 


By  (3.61)  it  is  clear  that  such  an  €3  >  0  exists.  We  prove  (3.78)  by  induction.  Suppose  that 
lz(i)l  ^  Dj  for  all  i  €  [k0.k— l].  Then  I  F(i.p(0(i)),z(i))  I  ^  Pf(Dj)  for  all  1  €  [k0.k— 1  ]:  hence. 
I  0(i)  I  <  00  for  all  i  €  [k0.k— l].  Letting  be  the  state  transition  matrix  of  z(i+l)  =  A(0(i))z(i)  it 


follows  from  Lemma  3.2  that  l<t>(n,.n2)l  <  K  X,n'  ^(e.Dj)  for  all  n,.n2  €  [k0.k].  Applying  the 


variation  of  constants  formula  to  (3.27).  we  have 


z(k)  =  <t>(k.k„)z(k0)  —  £  4>(k— l,i)G(i+l.p(0(i)).z(i))  . 


(3.80) 


Because  I  z(i)  I  ^  D,  implies  that  I  G(i+1.0(i).z(i))  I  <  ev^D,).  it  follows  from  (3.80)  that 


L"  *  A  A  1  I  -  K.V,pF(D,)  It -It. 


z(k)  I  <  KXr  °(€.D\)  I  z(k„)  I  +€ 


[1  "(e.D,)]  . 


(3.81) 


which  implies  for  e  €  [0.e2]  that  I  z(k)  I  <  D,.  Hence,  if  (3.78)  holds  for  all  i€[k„.k],  then  it  holds 


for  all  i€[k0.k+l].  Since  lz(k())l  <  D,.  (3.78)  holds  for  all  k ^ k„. 


Remark  3.5:  For  Lemma  3.5  it  is  sufficient  that  J'fk.S)  be  Lipschitzian  in  9  and  that  F(k.O.z)  be 


bounded  for  all  k  €  [k,,.oo)  .  9  €  0.  and  z  in  compact  sets.  That  is.  the  assumption  of  a 
Lipschitzian  derivative  of  v  and  the  assumption  that  F(k,0.z)  is  Lipschitzian  in  9.  z  can  be  dropped 


from  Assumptions  3.2  and  3.3.  respectively. 


Although  Lemma  3.5  shows  that  z(k)  converges  exponentially  to  a  ball  with  radius  0(e).  it 


does  not  show  that  \lf  is  exponentially  attractive.  In  order  to  establish  the  exponential  attractivitv 


ot  \L  we  introduce  the  deviation  of  z  from  h(k.@:e)  as  a  new  stale  variable 


T)  =  z  —  h(k.O;e) 


(3.82) 


Using  the  fact  that  h  satisfies  the  functional  difference  equaii 
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h(k+1.0  +  eF(k.p(0).h(k.0:€)):€)  =  A(p(0))h(k.0:e)  -  G(k+l.p(0),h(k.0:€))  .  (3.83) 

we  obtain  the  system  of  ordinary  difference  equations  for  (3.27)-(3.28)  in  7)  ,  0  coordinates 

7)(k+l)  =  A(p(0(k))>rT(k)-G'(k+1.0(k).T)(k);€)  (3-84) 

0(k+l)  =  0(k)  +  €F  (k.0(k).T}(k);€)  .  (3.85) 

where 


G  (k+1.0.7};€)  =  v(k+l  .p(0)  +  eF  (k.0.7))€)  —  i/(k+l,p(0)  +  eF  (k.0.O)€) 

+  h(k+l,p(0)  +  cF  (k.0,T>;€);e)  -  h(k+l.p(0)  +  eFU.O.O)*) . 


(3.86) 


F  (k.0,T):€)  =  F(k.p(0).h(k,0;e)  +  tj)  . 


(3.87) 


Remark  3.6:  We  could  not  use  7}  =  z— h  as  a  state  variable  in  the  continuous-time  case  in  Chapter  2 

because  we  did  not  prove  that  existed  and  7)  =  z  —  —  —  ^-0. 

00  9t  00 


With  the  help  of  Lemma  3.5  it  is  straightforward  to  show  that  r)  converges  exponentially  to 
zero  We  summarize  the  existence  and  attractivitv  results  in  the  following  theorem. 

Theorem  3.3:  Suppose  that  Assumptions  3. 1-3. 3  hold  and  that  (3.41)  holds.  Given  any  fixed 
A  €  (A.,,.1)  .  D()  >  0  .  A0  >  0  .  and  Dj  ^  D0.  let  p(0)  take  values  in  ©jte.D!).  There  exists 
€4(Do.Ao.Di.A)  >  0  such  that  for  each  €  €  [ 0,e4 )  (3.27)-(3.28)  possesses  an  integral  manifold  Me 
given  by  (3.16)  with  h(-.;e)  €  H((€/e2)D0.  (e/e2)A0).  where  €2  =  e2(D0.A0)  ^  e4  is  from  Theorem 
3.1.  Furthermore,  if  z(k),  0(k)  is  the  solution  of  (3.2  7 )-( 3.28 )  with  initial  data 
0(k„)  =  0,,  €  R^.  z(k,,)  =  Zo.  and  if  I  z„  I  ^  Dj/K.  then  z(k)  satisfies  (3.78)  and 

Tl(k)  =  z(k  )—h(k.0(k).€)  satisfies 

I  T)(k)  I  ^  K  I  tj( ku)  I  •  (3.88) 

Proof:  Choose  e4(D,„A,,.D, .A )  ^  minfe2(D,,,A,,)  •  €,(D.,.D,.A)}  such  that 

AjCej.D, )  +  e4  kp7(  I),  )[v,  +  (€4/e2)Ao]  ^  A.  ( 3.89) 


The  existence  of  \1«  follows  from  e4  ^  e2.  That  z(k)  satisfies  (3.78)  follows  from  €4  ^  €,.  With 
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I  z  I  -  I  h  +  Tf  I  ^  Dj.  we  bound  G'  by 

I  G'(k+l,0.7};e)  I  ^  e  +  (e/€2)A„)  t  ry  I  . 

Applying  the  variation  of  constants  formula  to  (3.84),  we  have 

T>(k)  =  <Kk.k0)7)(k„)—  £  4>(k— l.i)G'(i+1.9(i).7}(i);e) 

i=k0 


(3.90) 


(3.91) 


where  4>  is  the  same  state  transition  matrix  as  in  Lemma  3.5.  Taking  norms  and  applying  Lemma 
3.3  give 


I  T)(k)  I  <  KUjU.Dj)  +  e  Kp2(D,)(v,  +  (e/€2)A0)]k_lc°  I  7)(k„)  I  . 
which,  in  light  of  (3.89).  completes  the  proof. 


(3.92) 


We  illustrate  the  use  of  Theorem  3.3  in  an  example.  First  we  rewrite  (3.85)  as  a  perturbed 
version  of  (3.77) 


e(k+l)  =  0(k)  +  €F(k.p(0(k)).h(k.0(k);e)) 

+  €[F'(k.e(k).7)(k);€)  -  F'(k.0(k).O;e)] 


(3.93) 


where  the  perturbation  is  exponentially  decaying  to  zero.  For  simplicity  we  consider  the  case 
where  0  =  0  is  an  equilibrium  of  (3.77).  B(C,O)£0.  and  for  each  €  €  [0.€4(D().Ao.D,  A))  we  have 

I  0  +  €F(k.0.h(k.0;e))  I  <(l-«X3)l0l.  ¥  0  €  B(C.O)  .  (3.94) 


where  A2  €  [O.p/0.0)).  Note  that  this  implies  I  p(0)  +  eF(k.p(0).h(k.0;e))  I  ^  I  p(0)  I  ;  hence,  we 
can  take  0i(e.D)=B(C,O).  Note,  also  that  (3.94)  is  almost  never  satisfied  with  A2  >  0  in  adaptive 
systems  of  the  type  (3.1 )-( 3.2).  However,  the  basic  idea  does  not  change  if  the  right-hand  side  of 
(3.94)  is  changed  to  ( 1—  €A2)|0|+€S  where  5€[0.A,C). 

Because  (3.94)  implies  that  0  =  0  is  a  uniformly  stable  or  an  exponentially  stable  equilibrium 
oi  <3  77)  and  because  the  perturbation  in  (3.93)  is  exponentially  decaying  to  zero,  it  follows  that 
0  =  o  is  a  uniformly  stable  or  an  exponentially  stable,  respectively,  equilibrium  of  (3.93). 
There! ore.  the  solution  rj  =  0  .0  =  0  of  (3.84 )-( 3.85)  is  uniformly  stable  if  A,  =  0  and 


(j 
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exponentially  stable  if  X2  >  0.  The  interesting  problem  is  to  estimate  the  region  of  attraction  of 
this  solution.  Letting  rn(k)  satisfy 


r^k+l)  =  Ar^k)  .  r(k0)  =  Klr^ko)!  . 


(3.95) 


it  is  clear  that  I  i)(k)  I  ^  r„(k)  for  all  k  ^  k0  for  each  €  €  [O.e^Do.Ao.D^A)).  Letting  r@(k)  satisfy 


r#(k+l)  =  (1— eX2)  re(k)  +  epjCDj)  r,,(k)  .  r6(k0)  =  t  0( k0)  I  . 


(3.96) 


it  follows  that  if  0(i)  €  B(C,0)  for  all  i  €  [k0.k—  l],  then  0(k)  €  B(C.O).  Thus,  we  can  estimate  the 
region  of  attraction  by  finding  conditions  such  that  the  solution  of  the  linear  time-invariant  system 
(3.95)-(3.96)  keeps  r#(k)  ^  C  for  all  k  ^  k0.  Taking 


V(rfl.r  -e)  =  r#  + 


epz(Di) 

1 — eA->— A 


(3.97) 


we  have 


V(r«(k+l).rT,(k+l):e)  =  ( 1-€X2)  V(rs(k).  r„(k):€) 

=  (l-eA,)k~k°  V(r„(k0).  rJk,,):*)  . 


(3.98) 


which  proves  the  following  corollary.  Because  we  are  proving  0  6  0  for  all  k  ^  k,„  we  can  state 
the  result  directly  for  the  original  system  (3.1 )-( 3.2 ). 

Corollary  3.2:  Suppose  that  Assumptions  3. 1-3. 3.  (3.41),  and  (3.94)  hold,  that  €  <  €4(D0.Ai>.D|.X) 
and  that  B(C.O)  Q  0.  Let  x(k),  0(k)  be  the  solution  of  (3.1 )-( 3.2 )  with  initial  data 
x(k,,)  =  x,,  .  0(k„)  =  0„.  If  I  x0— r(ko.0„)  I  ^  D]  and 


V0  =  V(I0(,I  .  K  I  x„  —  g(k().0(l:6)  I  :« )  <  C. 


(3.99) 


xf k ) — g(k.0(k):e)  I  ^  KAk  k°  I  x„— g(k,,.0,,;e) 


0(  k )  I  <  KV„(l-€X,)k  k".  (3.100) 


In  many  algorithms  the  dependence  of  f(k.0.x)  on  x  is  quadratic  or  higher  order.  Then  there 
exist  p,(  I),,)  and  p4( D] )  such  that 
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pz(  I  2  I  )  <  p3(D„)  +  p4(D,)  I  7) !  .  (3.101) 

In  such  cases  we  can  arrive  at  a  less  conservative  estimate  of  the  region  of  attraction. 

Corollary  33:  In  Corollary  3.2.  if  (3.101)  holds,  then  V(r@.rT);€)  given  by  (3.97)  can  be  replaced 

by 


V(re.rt);€)  =  re  + 


p3^n) 

€  1-€A2-A  rn 


+  € 


P4(Pl) 

1 — eX.2- A.2 


(3.102) 

□ 


Thus,  we  have  shown  that  the  possession  of  an  exponentially  attractive  integral  manifold  Me 
by  (3.27M3.28)  implies  that  the  full-order  system  has  the  property  that  the  existence  of  a 
uniformly  (asymptotically)  stable,  an  exponentially  stable,  or  an  unstable  solution  of  the 
reduced-order  system  in  Me  (3.77)  implies  the  existence  of  a  uniformly  (asymptotically)  stable,  an 
exponentially  stable,  or  an  unstable  solution  of  the  full-order  system  (3.27M3.28).  respectively. 
Furthermore,  if  this  solution  of  (3.77)  lies  in  the  interior  of  9.  then  the  corresponding  solution  of 
the  modified  system  (3.27)-(3.28)  is  transformed  via  x(k)  =  J^(k.0(k))  +  z(k)  into  a  solution  of  the 
original  system  (3.1 )-( 3.2 ).  In  the  next  section  we  study  via  averaging  the  behavior  of  solutions  of 
(3.77).  We  conclude  this  section  with  a  reminder  that  the  assumptions  under  which  these  results 
were  derived  are  very  mild.  This  is  especially  true  on  the  inputs  to  the  system,  namely,  w(k)  and 
the  k  dependence  of  f(k.0.x).  which  are  only  required  to  be  uniformly  bounded. 
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3-5.  Analysis  in  the  Manifold:  Averaging 

The  system  (3.1 )-( 3.2)  restricted  to  M(  behaves  according  to  (3.4)  which  is  in  the  standard 
Bogoliubov  form  for  the  method  of  averaging.  However,  the  literature  on  the  method  of  averaging 
for  discrete-time  systems  with  deterministic  inputs  is  almost  nonexistent.  Meerkov  (1973) 
presents  elegant  proofs  using  simple  mathematics  of  several  standard  averaging  theorems  tor 
continuous-time  systems.  He  then  stales  the  corresponding  theorems  for  discrete-time  systems 
pointing  out  that  the  discrete-time  proof,  which  is  not  given,  is  virtually  a  copy  of  the  continuous- 
time  proof.  As  the  hypotheses  of  the  theorems  provided  by  Meerkov  are  somewhat  different  than 


the  hypotheses  which  our  system  satisfies,  we  shall  state  and  prove  several  basic  theorems  from  the 
method  of  averaging  Our  proofs  are  modelled  on  Meerkov's  but  our  theorem  statements  are  in  the 
style  of  Sethna  and  Moran  (1968).  While  the  literature  on  the  method  of  averaging  for  discrete¬ 
time  systems  with  deterministic  inputs  is  scarce,  there  are  many  references  which  relate  the 
behavior  of  (3.4)  or  (3.1  )-(3.2)  with  stochastic  inputs  to  the  behavior  of  the  ODE  (3.5): 
Derevitskii  and  Fradkov  (1974).  Ljung  (1977).  Kushner  (1977).  Kushner  and  Clark  (1978). 
Benveniste,  Goursat.  and  Ruget  (1980),  Benveniste.  Ruget  (1982),  Kushner  and  Swartz  (1984). 
Metivier  and  Priouret  (1984).  just  to  mention  a  few.  We  conclude  this  section  by  showing  that 
some  with  probability  one  results  relating  the  behavior  of  (3.1M3.2)  to  that  of  the  ODE  (3.5)  can 
be  stated  as  corollaries  to  our  basic  averaging  theorems.  We  feel  that  our  approach  of  giving  a 
complete  deterministic  proof  and  then  adding  stochastic  assumptions  offers  the  simplest 
introduction  to  this  area,  and  is  at  least  of  pedestrian  interest. 

In  order  to  simplify  the  appearance  of  the  equations,  we  shall  make  a  few  notational  changes. 
We  assume  that  constants  D0  >  0  .  A0  >  0  .  Dj  ^  D0  .  and  A  €  (X0.l)  have  been  chosen.  We  let 
€4  =  €4(D0.A0.Di.A)  .  pF  =  pF((€4/€2)  D0).  and  Pi  =  Pi((e4/€2)  D0  .  (e4/€2)A0)  where  €2  =  e2(D0.A()). 
Finally,  we  use  f(k,0;e)  to  denote  f(k.0,g(k.0;e))  =  F(k.0.h(k.0;e))  and  we  define 

p(  =  ( l/€2)  DoPz((€4/e2)  D0).  It  follows  that  for  each  €  €  [0.€4).  every  0.0  €  0.  and  all  k  €  Z 

I  f(k.0;e)  I  <  pF.  I  f(k,0,e)— f(k.0  :e)  I  <  p,  I  0-0  I  .  I  f(k.0;e)-f(k.0;O)  I  <  ep{ .  (3.103) 

As  we  are  interested  in  the  behavior  of  solutions  of  (3.4)  or  (3.77)  only  for  0  €  0.  we  do  not 
need  the  projection  p(0):  hence,  we  study 

0(k+l)  =  0(k)  +  €f(k.0:e).  (3.104) 


We  use  the  classical  notation  0(k:0<t.k„)  to  represent  the  solution  of  (3.104)  with  initial  data 
0(k„)  =  0„. 

Assumption  3.4:  The  limit  on  the  right-hand  side  of  the  definition 
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a  i  k+N-1 

f  (0)  =  lim  i  V  fCi.e.O)  (3.105) 

N-oo  1\  ~ 

exists  uniformly  with  respect  to  k  €  Z  and  0  €  9.  That  is.  there  exists  a  strictly  increasing 
continuous  function  k(  )  with  k(0)  =  0  such  that 

If(fl)-^  £  f(i.0:O)l  <  Kfi)  (3.106) 

N  1=lc 


for  all  k  €  Z  and  0  €  8. 


□ 


Remark  3.7:  If  f(k.0;O)  is  N-periodic  then  we  define  f  by 

T(0)  =  1  z!  f(>.0:O) 
|=0 

and  we  take  k(^)  =  0  in  the  following  derivations. 


(3.107) 


□ 


i 
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We  relate  solutions  of  (3.104)  to  solutions  of  the  ODE 

—  S'  *  7(5")  .  (3.108) 

dr 

We  use  the  notation  &(t:90)  to  represent  the  solution  of  (3.108)  with  initial  data  W(0)  =  0„.  (Since 
(3.108)  is  time-invariant,  there  is  no  loss  of  generality  in  taking  r=0  as  the  initial  time  in  (3.108).) 
Because  f  represents  the  average  of  f.  (3.108)  is  also  referred  to  as  the  averaged  system.  Our  tool 
for  establishing  relationships  between  the  trajectory  of  (3.104).  0(k;0().ko),  and  the  trajectory  of 
(3.108)  sampled  at  tk  =  e(k— k„)  .  0"( tk;0,i).  is  the  averaged  trajectory  defined  by 

0(k:0„.k„)  =  4"  "  Z  *  ^o.k,,)  .  (3.109) 

'  i=k 

where  N  is  to  be  determined  and  is  possibly  a  function  of  €.  The  averaged  trajectory  0(k;0,„k,>)  is 
simply  a  moving  average  of  length  \  of  the  trajectory  0(i:0,,.ko)  over  a  window  beginning  at  i  =  k. 
We  make  use  of  the  average  trajectory  in  a  two-step  procedure.  First,  we  bound  the  distance 
between  0(k:0„.k,,)  and  0(k;0<„k„)  and  second,  we  bound  the  distance  between  0(k:0,,.k,,)  and 


■sfis 
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5’(tk:do).  Then  the  triangle  inequality  gives  us  a  bound  on  the  distance  between 
0(k:0o.ko)  and  9Xtk:0o). 

Lemma  3.6:  Suppose  that  0(k:0o,ko)  6  0  for  all  k  €  [k0.kj+N— l].  Then,  for  each  €  €  (0.e4)  the 
distance  between  the  original  and  averaged  trajectories  is  bounded  by 


0(k:0o.ko)  -  0(k:0o.ko)  I  <  (€N-e)(pF/2) 


(3.110) 


for  all  k  €  [ko.kiJ. 

Proof:  Using  (3.103)  we  have  the  bound  for  i  €  [k0.ki+N— l] 


I0(i+l;0o.ko)-0(i:0o.ko)l  <  €Pf 


(3.111) 


which  implies  that  for  k  €  [k0.kj  and  i  €  [k.kj+N— l] 


0(i:0o.ko)  —  0(k;0o.ko)  I  ^  epF(i-k) 


(3.112) 


From  the  definition  of  0  and  the  triangle  inequality  we  get 


1 0(k:0o.ko)  0(k;0o.ko)  I  ^  —  £  1 0(j:0(),ko)  0(k.0o.kn)  I 

i=k 

.  k+N-1 

^  E  epF(i— k) 

*'1  i=k 


€Pf  N(N— 1) 


=  (€N-€)(pF/2) 


for  all  k  €  [k„.k|]. 


(3.113) 


Lemma  3.7:  Suppose  that  €  9  for  all  r  6  [O.rJ.  Let  k](e)  =  [r,/€].  that  is.  the  largest 

integer  less  than  or  equal  to  Tt/e.  Then,  for  each  €  6  (0.e4)  the  trajectory  0”(t:0())  of  the  ODE 
(3.10M  sampled  at  tk  =  e(k— k,,)  satisfies  the  ordinary  difference  equation 


SUk+1)  =  ff(tk)  +  €f(JT(tk))  + f,(IF(tk):€) 


(3.114) 


for  all  k  €  [k,,.k,,+k,(€)— 1  ].  Furthermore,  fj  defined  by 


f,(0;e)  =  y(€:0)-(e  +  ef(0)) 


(3.115) 


m&mm 
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satisfies 


f1(0‘(tk:0o):€)  I  <  €2CplPF/2) 


(3.116) 


ii  £ 


c  i 


for  all  k  6  [k0.k„+k,(e)-l]. 

Proof:  It  is  clear  from  the  time-invariant  nature  of  (3.108)  and  tk+i- tk  =  €  that 

9"( tfc+1  ;0o)  =  ffie^ftk^o)):  hence.  (3.114)  follows  from  the  definition  (3.115).  To  establish  (3.116) 
we  let  0  denote  ^(t^Oo)  for  any  k  6  [k0.k0+ki(e)— l]  and  compute  the  bound 


i  ?(€:#)— 0— €7(0)  l  =  I  /  (T(0‘(r:0))-T(0))drl 

0 

€ 

^  /  Pi  I  5Xt:0)  —  0  I  dr 

0 

€  T 

=  /p,!0  +  /T(F(s:0)))ds-0ldf 

0  0 

€  r 

^  PiPf  J  f  dsdr  =  €2(p!pi:/2)  . 


(3.117) 


p  £ 
L". 


Lemma  3.8:  Suppose  that  Assumption  3.4  holds  and  that  0(k:0o.k„)  €  0  for  all  k  €  [ku.k1+N  l]. 
Then,  for  each  €  €  (0.e4)  the  averaged  trajectory  0(k;0,i.k„)  satisfies  the  ordinary  difference  equation 


0(k+l)  =  0(k)  +  ef(0(k))  +  f2(k:0o.ko.€) 


(3.118) 


•i 


for  all  k  €  [ku.k]— l]  where  f2  satisfies 


f2(k;0„.ko.€)  I  ^  e  k( ^)  +  €(€.\-e)piPF  +  e2p{ 


(3.119) 


pi  s? 
K  S’ 


for  all  k  €  [ k,,.K | —  1  ]. 

Proof:  From  the  definition  of  0  it  follows  that  for  all  k  €  [k„.ki— l] 


t 


0(k+l:0o.ko)-0(k:0o.ko)  =  ±  (0(k+N;0o.ko)  -  0(k:0o.k„)) 

t  N_1 

=  -^r  £  €f(k+i.0(k+i;0o,ko):€) 

'  i=0 


]  N-l 

=  lu  Z  €f(k+i.0(k:0o.ko);O) 


»i  **  V*'  ■  ••vvn,wy,i\y/,v/ 

1=0 

«  N-l  (3.120) 

+  Tf  Z  Uf(k+i.0(k+i:0o.k„):O)  -  ef(k+i:0(k:0n.ko):O)] 

1=0 

1  N-i 

+  M-  z  [€f(k+i.0(k+i:0o>ko):€)  -  €f(k+i.0(k+i:0o.ko):O)] 


=  €f  (0(k:0o.ko))  +  f2(k;0o.k>o.e) 
where  f2  =  f21  +  f22  +  f23  +  f24  with 


f2i(k;0o,ko.€)  — 

1  n-i  . 

—  £  ef(k+i.0(k;0o.ko);O)  -  eT(0(k:0o.ko)) 

^  i=() 

(3.121) 

f22(k;0(|,ko.€)  = 

1  N"1  /  \ 

—  £  ef(k+i.0(k:0o.k„):O)  -  ef(k+i.0(k;0o.ko):O) 

iN  i=0  '  1 

(3.122) 

f23(k  :0(l.k(|.e)  = 

1  N_1  f  i 

—  £  €f(k+i.0(k+i;0ft.k„):O)  —  €f(k+i.0(k:0o.ko):O) 

1=0  *  1 

(3.123) 

f  24(  k  ;0(,,ko.€ )  = 

i  N-I  (  . 

—  £  ef(k+i.0(k+i:0o.ko):€)  -  €f(k+i.0(k+i:0o.ko):O)  . 

1=0  '  J 

(3.124) 

By  Assumption  3.4  f21  is  bounded  by 


f21(k;0„.k,,,€)  i  ^  ck(A-) 


(3.125) 


Lsing  Lemma  3.6  it  follows  that 


f22(k:0,,.ko.€)  I  ^  —  £  ep,(€\— e)(pF/2) 
A  i—o 

=  €(e\-€)(pipf./2)  . 


(3.126) 


Similarly  to  the  proof  of  Lemma  3.6  we  obtain  the  bound  for  f » 


f23(k:01,.k„.€)  i  <  —  £  €p,€pFi  =  e(€\— €)(p,pF/2) 


(3.127) 


L  i  na  1 1  y  we  bound  f24  bv 
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lf24(k:0o.ko.€)l  <  4  *  €V«  •  (3.128) 

The  triangle  inequality  and  (3.125)-(3.128)  imply  (3.119) 

□ 

Thus,  we  have  shown  that  0"  and  0  are  both  solutions  of  ordinary  difference  equations  which 
are  perturbations  of 

0(k+l)  =  0(k)  +  e7(0).  (3.129) 

From  (3.116)  it  is  clear  that  the  perturbation  f!  in  (3.114)  can  be  made  arbitrarily  small  with 
respect  to  ef  by  taking  e  sufficiently  small.  If  we  take  N  =  N(e)  =  e_r  for  r  €  (0.1)  it  follows  from 
(3.119)  that  the  perturbation  fi  in  (3.118)  can  also  be  made  arbitrarily  small  with  respect  to  €f . 
Notice  also  from  (3.110)  that  this  choice  of  N'(c)  allows  us  to  make  the  distance  between  0  and  9 
arbitrarily  small.  Hence,  with  these  three  lemmas  we  can  generate  many  results  relating  solutions 
of  the  ordinary  difference  equation  (3.104)  and  the  ODE  (3.108).  We  present  first  a  result  on  finite 
time  approximation. 

Theorem  3.4:  Suppose  that  Assumption  3.4  holds.  Given  any  positive  constant  Tj  <  co.  no  matter 
how  large,  and  any  <x  >  0,  no  matter  how  small,  if 

B(o-.W(t:9J)  Q  0  (3.130) 


for  all  r  €  [O.rJ.  then  there  exists  €j(T,.cr)  €  (0,e4]  such  that  for  each  €  €  (O.Cj)  and  any  k„  6  Z 


1  0(k:0„.k,,  —  FUuifl,,)  I  <  cr 


(3.131) 


for  all  k  €  [ k, ..k, k t (e )]  where  k,(e)  =  |T|/ej  and  tk  =  e(k— k,,). 

Proof:  Let  €5(r , .cr )  be  the  smaller  of  e4  and  the  solution  of 

<7  =  €  (pF/2H3  +  e"1  ')  +  </c(€  )  +  €  p,pF  +  ept)  -  (3.132) 

P\ 

I  et  \  =  \(e )  =  \e~  |.  We  prove  (  3. 1  3 1 )  by  induction.  Suppose  that 


0(k:0„.k„  -  9"(ik:0„) '  <  cr  -  e  p,. 


(3.131) 
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holds  for  all  k  €  [k0.i]  for  some  i  €  [ko.ky+k!— l].  Then  Bfe^Pp.fKi'.Oo.ko))  £  0  which  implies 
0(k;0o.ko)  €  0  for  all  k  €  [k0.i+N].  From  this  it  follows  that  0(k;0o.ko)  is  well  defined  for  all 
k  €  [krt.i+l].  Using  Lemmas  3.6-3.8  we  have  for  all  k  €  [k0.i] 

I0(k+l:eo.ko)-0'(tk+1:0o)l  <  (1  +  €pt)  1 0(k:0o.ko)— FU^o)  I 

+  e(n(e*)  +  e^-ptPf-eipxPf/D  +  ep€) 

^  (1 +€p1)k+1_k°€*(pH/2) 


+  (7r(eVl)  +  e^pjPf-eCpiPF/l)  +  epe) 


(l+€pj)k+1  k°-l  (3.133) 


<  <r-ie*pF. 


Lemma  3.6.  the  triangle  inequality  and  (3.133)  imply  that  (3.131  )  holds  for  all  k  €  [k().i+l]. 
Hence,  if  (3.131)  holds  for  all  k  €  [k0,i]  for  any  i  6  [kft.k()+ki— 1],  then  it  holds  for  all 
k  €  [k„.i+l].  Since  0(k„:0(,.k„)  =  0„  =  0"(O:0o)  which  guarantees  that  (3.131')  holds  for  k=kp. 
(3.131')  holds  for  all  k^k„. 


Remark  3.8:  We  can  get  a  larger  estimate  for  e5  by  letting  N  =  e~r,  defining  €3(7s1,cr.r)  as  the 
solution  of 


cr  =  e1  r(pF/2)(3  +  ep,T|)  +  (tcfe1  r)  +  e1  rPjPF  +  ep€) 


ep'r‘-l 


(3.134) 


and  taking  €s(r,.cr)  =  minie4.max  ie5(T,,o\r)}}.  The  proof  using  this  estimate  requires  a  change  on 

r€  [0.x | 

the  right-hand  side  of  (3.131)  from  cr  —  e‘/:pF  to  O’  —  61  r,pF  where  r.  €  [O.l]  is  the  value  for  which 

e,  attains  its  maximum.  In  general,  the  bounds  provided  by  Lemma  3. 6-3. 8  are  so  conservative 
that  this  procedure  still  results  in  a  very  conservative  bound  for  e5. 


Remark  3.9:  It  f(k.0:<>)  is  \-periodic.  then  we  replace  (3.132)  by 
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cr  =  eN(p?/2)(3  +  ep,r')  +  (eNpiPp  +  ept) 


e  1  *— 1 


(3.135) 

□ 


In  order  to  give  a  more  complete  connection  with  the  ODE  literature  in  the  stochastic  setting. 


we  consider  the  system 


0(k+l)  =  0(k)  +  a*  f(k,0(k):ak)  .  k  ^  0  , 


(3.136) 


where  ak  is  a  monotonically  decreasing  sequence  which  satisfies 


lim  ak  =  0  .  ak  >  akH  ^  ak  —  ak2i  .  V  i  ^  k^O 

k— oo 


(3.137) 


We  note  that  ak  =  (k+l)  r  for  r  €  (0.1  ]  satisfies  (3.137).  Letting  9  (k;0|(.k„)  denote  the  solution  of 


(3.136)  with  initial  data  6  (k0)  =  90.  we  have  the  following  corollary  to  Theorem  3.4. 


Corollary  3.4:  Suppose  that  Assumption  (3.4)  holds  and  that  (3.103)  holds  for  all  €  6  [0,e4). 


Given  positive  constants  r j  and  cr,  if  (3.130)  is  satisfied  for  all  r  €  [O.tJ.  then  there  exists 


VKr^cr)  such  that  for  any  k„  ^  M 


I  6  (k.60.k0)  —  9"(tk;0o)  I  <  <r 


(3.138) 


for  all  k  €  [ko.ko+k^e)]  where  tk  =  £  and  kj(e)  is  the  largest  integer  such  that  t  ,  ^  r,. 


Proof:  Clearly  the  bound  (3.110)  in  Lemma  3.6  holds  with  9  replaced  by  9  and  6  replaced  by  ak(). 


Likewise,  in  the  difference  equation  (3.114)  in  Lemma  3.7,  we  replace  lk  +  J  by  tk+1  .  tk  by  tk.  and 


by  ak.  Hence,  in  the  bound  (3.1 16)  we  replace  €  by  ak().  In  Lemma  3.8  we  let  9  be  the  average  of 


9  .  redefine  f 22  f 21  ■  and  f:4-  add  another  term  loS  to  .  and  replace  €f  (0(k))  by  akf  (0(k)).  In  the 


term  f 2 ]  we  replace  c  by  ak  ^  ak|j.  Because  we  changed  e  to  ak  and  not  Qk+1  in  f21  we  must  add 


another  term  f:5  to  f\. 


r2S(k:eil.k,1)  =  £  <ak+,-ak)  r(k+i.0(k:0,,.k,,);O) 

A  1=11 


(3.139) 


1  sing  (3.137)  we  bound  f2<;  bv 
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f25(k;0o.ko)  I  ^  1.  £  PfaZo  i  =  ako(ak#N  -  ako)(pF/2) 


N 

i=0 


(3.140) 


Replacing  the  bound  (3.119)  with 


f2(k:0o.k„)  I  ^  ako(>c(  A.)  +  (akoN-ako)(p,  +  i- )pF  +  akpt) 


(3.141) 


the  proof  is  completed  as  in  the  proof  of  Theorem  3.4  with  M(r1.a‘)  chosen  large  enough  so  that 
aM  <  e4  and 


cr  ^  a^(pi:/2)(3+ePiri)  4- (x(avf)  +  ckm(Pi  +  y  )Pf  +  aM(pt+pF/2)) 


Pi 


(3.142) 


Remark  3.10;  As  in  Remark  3.8  we  can  get  a  smaller  estimate  of  M(r.cr)  by  letting  N  =  a^'  and 
optimizing  with  respect  to  r.  The  estimate  can  also  be  improved  if  a  specific  ak  sequence  is  chosen 
or  a  decaying  upper  bound  is  used  in  (3.137). 


□ 


Before  giving  results  on  infinite  time  approximation  of  0(k)  by  (T(tk),  we  make  several 
observations  about  the  finite  time  results  which  we  just  presented.  The  conditions  under  which 
Theorem  3.4  and  Corollary  3.4  are  established  are  very  mild,  namely,  that  f  is  bounded  and 
Lipschitzian  in  9  and  €  and  that  the  average  f  exists.  We  also  emphasize  that  the  same  conditions 
are  required  for  the  constant  gain  case.  (3.104).  and  the  decaying  gain  case.  (3.136),  and  that  the 
same  ODE.  (3.108).  is  associated  with  the  constant  gain  case  and  the  decaying  gain  case.  The 
difference  between  the  two  cases  is  that  for  the  constant  gain  case  we  sample  the  ODE  periodically 

/  k-1 

at  tk  =  e(k— k„).  whereas  in  the  decaying  gain  case  we  sample  the  ODE  at  times  u  =  £  a,  which 


are  closer  together  as  k  increases.  The  price  we  pay  for  not  using  more  information  about  f  is  that 


the  approximation  of  0(k:0,,.k,,)  by  F(tk;0,,)  and  the  approximation  of  0  ( k :0,,.k,,)  by  tk :0, , )  are 
guaranteed  only  lor  a  finite  time  interval.  In  fact.  even  if  (3.130)  holds  for  all  r  ^  0.  the 
approximations  do  not  hold  for  all  r  0.  For  example.  0,,  could  be  an  unstable  equilibrium  of 
(3.108)  with  the  property  that  W(r:9])  reaches  the  boundary  of  0  in  finite  time  lor  every  0j  ^  On 


vZpa 


If  f(k.0o;€)  is  not  identically  equal  to  zero,  then  for  some  k[  >  k„  ,  0(k:0,,.k„)  =  01  5*  0O.  Hence,  for 
e  sufficiently  small  and  k  ^  kj.  flCkifly.ko)  =  0(k.0i.k)  follows  ^(r:9I)  to  some  neighborhood  of  the 
boundary  of  ©  in  finite  time  which  implies  0"( tk :©o)  —  $<> is  not  a  good  approximation  of  0(k;0o.ko) 
for  all  k  ^  k0. 

Clearly  we  must  make  some  additional  assumptions  about  f  (0)  or  the  solutions  of  the  ODE 
(3.108)  in  order  to  be  able  to  relate  5"(tk;0o)  and  0(k;0„)  over  infinite  intervals.  We  shall  consider 
two  different  assumptions.  For  continuity  with  respect  to  the  deterministic  averaging  theory,  we 
prove  an  infinite  time  result  under  the  assumption  that  the  ODE  has  an  asymptotically  stable 
constant  solution.  For  continuity  with  respect  to  the  literature  on  the  ODE  method  in  adaptive 
systems  and  for  ease  of  application  in  Chapter  4,  we  prove  an  infinite  time  result  under  the 
assumption  that  a  Lyapunov  function  with  certain  properties  exists. 

We  let  9.  in  the  interior  of  0  be  an  asymptotically  stable  equilibrium  of  the  ODE  (3.108)  and 


denote  the  region  of  attraction  by  0a  . 


0A  =  \9„  €  0  :  2T(t;0o)  €  0  ¥r  ^  0  and  lim  &(t:90)  =  0.}  . 


(3.143) 


As  in  Theorem  3.4  we  need  a  ball  around  W  to  be  in  0:  hence,  we  define  a  subset  ©a^0- )  of  ©A. 


©a(<t)  =  (0„  6  0A  :  Bfcr.ERr;©,,))  £  0  ¥r  ^  0)  . 


(3.144) 


Theorem  3-5:  Suppose  that  Assumption  3.4  holds  and  that  0.  in  the  interior  of  0  is  an 

/ 

asymptotically  stable  equilibrium  of  the  ODE  (3.108).  Given  any  c  >  0  for  which  0A(cr)  is  not 
empty,  there  exists  ef,(cr)  6  (O.eJ  such  that  for  each  6  €  (0.eh),  any  k,,  €  Z  and  any  0,,  €  &A(cr) 


0(k:0„.k())  -  0"(  tk :©,,)  I  <  cr 


(3.145) 


tor  all  k  ^  k...  w  here  tk  =  e(  k— k, , ) . 

Proof:  The  proof  follows  Meerkov  (1973).  We  use  the  following  two  consequences  ol  the  tact 
that  0.  is  an  asymptotically  stable  solution  of  the  ODE  (3.108).  Let  a<,.  >  <>  be  chosen  so  that 


B(  )  Q  0  x(  cr  ).  There  exists  r.(  fj. )  <  oo  such  that  lor  each  //  €  ( o./tt,,]  and  tor  every  0,,  €  0  x(cr  i. 


we  have 


I  JF(t:0<,)  —  0. 1  <  m/2  (3.146) 

for  all  r  ^  t.(m).  There  exists  a  continuous  function  8(m)  with  8(0)  =  0  such  that  for  each 
M  €  (O.Mo]  and  for  every  0j  with  I  0!  —  0. 1  <  m 

I  0"(t;0j)  —  0. 1  <  8(m)  (3.147) 

l 

for  all  t  ^  0.  Letting  €5(m)  =  e 5(t.(m)  +  €:m/2).  we  have  for  each  m  €  l(O.Mo)  O  (0.2cr)}  and  any 
k0  6  Z 

I  0(k:do.ko)  —  0‘(tk:0o)  I  <  m/2  (3.148) 

for  all  k  6  [ko.kj  where  kj  =  k^M.c)  =  [t.(m)/€ |  +  1  and  tk  =  e(k— k0).  Hence,  from  the  triangle 
inequality.  (3.146).  and  (3.148)  we  have 

I  0(k:0(|.k(i)  —  0.  I  <  m  (3.149) 

at  k  =  k0+kj.  Now  we  consider  the  interval  [k0+k1.ko+2k1].  Let  0]  =  0(ko+kj:0„.k„).  NVe  note 
that  0(k:0().k„)  =  0(k:01.k„+k1)  for  k  ^  k„+ki  and  that  I  0,-0. 1  <  m-  Applying  Theorem  3.4 
again,  we  get 

l0(k:0o.k„)-5'(tk-tIC|;01)l  <  m/2  (3.150) 

for  all  k  €  [k„+k,.k,,+2k1].  We  point  out  that  in  (3.150)  we  are  comparing  0(k:0o.ko)  with  a 
different  trajectory  of  the  ODE  than  in  (3.148).  Applying  the  triangle  inequality.  (3.147).  and 
(3.150).  we  have 

l0(k:0„.k„)-0. 1  <m/2  +  8(m)  (3.151) 

for  all  k  €  [k,,  +  k  1.k,,+2k  J.  Furthermore,  from  the  definition  of  r.  and  k[  we  have 

0'(it-iki:0,)-0.  I  <  m/2  (3.152) 

at  k  =  k,.  +  2k,;  hence.  (3.149)  holds  at  k  =  k,,+2k|.  We  prove  by  induction  that  (3.151)  holds  for 
all  k  ^  k.  +k|  and  that  (.3.149)  holds  at  k  =  k,,+nk,  for  all  integers  n  ^  1  which  implies  that 


(3.151)  holds  for  all  k^k0+ki.  Suppose  that  (3.149)  holds  at  k  =  ko+nk,  for  any  integer  n  ^  1. 
Letting  0j  =  0(k()+nk:0o.ko)  it  follows  that  (3.150)  holds  for  all  k  6  [k0+nk1.k0+(n+l  )kj  with 
tic  ~  4,  replaced  by  tk  —  t^.  Then  the  triangle  inequality.  (3.150).  and  (3.147)  imply  that  (3.151) 
holds  for  all  k  6  [k0+nk,.k0+(n+l)k1+k1].  Clearly.  (3.152)  holds  at  k  =  k0+(n+l)k!+ki  with 
tic-tk,  replaced  by  tk-tnki:  hence.  (3.149)  holds  at  k  =  k0+(n+l)+k,.  Since  (3.149)  holds  at 
k  =  kn+kj.  it  follows  that  (3.149)  holds  at  k  =  k0+nk,  for  all  n  £  1  and  (3.151)  holds  for  all 
k  ^  ko+kj.  One  more  application  of  the  triangle  inequality  gives 


|0(k:0o.ko)-'9(tk:0o)|  <  |0(k;0o.ko)-0.|  +  |3(tm;0o)-0.|M+8(/i)  (3.153) 

for  all  k^ko+kp  Then  choosing  /x1(cr)€(0,Ato]  such  that  /i1+%,)^or  and  taking  €6(cr)=€5(//1) 

complete  the  proof. 


□ 


Corollary  3.5:  Suppose  that  Assumption  3.4  holds,  that  (3.103)  holds  for  all  e6[0.e4).  and  that  0. 
in  the  interior  of  0  is  an  asymptotically  stable  equilibrium  of  the  ODE  (3.108).  Given  any  cr>o 

for  which  0A(o-)  is  not  empty,  there  exists  M,(cr)<oo  such  that  for  any  k„^M  and  any  0o€0A(tr) 

|0  (k;0(>.ko)  — U(tk;0o)|  <  o-ak^>F  (3.154) 

,  k— 1 

for  all  k^k,,.  where  tk  =  £  a,.  Furthermore. 

>=*o 


lim  0  (k:0„  ’  =  0,  .  (3.155) 

k  —  oo 

Proof:  In  order  to  establish  (3.154)  we  follow  the  proof  of  Theorem  3.5.  except  that  the  repeated 
applications  of  Theorem  3  4  over  intervals  [k,p+nki,k(,+(n+ 1  )k ,]  are  replaced  by  repealed 

applications  of  Corollary  3.5  over  intervals  [k„+k,  .k,,+kJ+I]  where  k,  is  chosen  so  that 

.  k  I  —  I  l  _  i 

’  •  /  ,  kn • I  1 

1  ■  =  L  0'i€[r.(^),r. tjul+o  ,]  and  k.,»,  is  chosen  so  that  l  ,  —  t  ,  =  ]F  a ,  €[r.(//).r.(/u)+a  .  ] 


lor  n  ^  1.  We  note  that  (3.137)  implies  that  lim  £  a,  =  oo  which,  in  turn,  implies  that  k„<oo  lor 
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each  integer  n€[0.oo).  To  see  that  (3.155)  is  true  we  point  out  that  in  the  proof  of  (3.154)  we  have 

i  l  * 

shown  that  for  any  k0^Mi(cr)  and  any  0o€0A((r).  |0  (k:0o.ko)— 0.|  <  <r  for  all  k^ki-  Given  any 

/  iiii  I 

a  €(0,<r)  it  follows  that  0  (k;0o.ko)€0A(<r  )  for  all  k^k^  Choosing  i0  so  that  io^Mi(cr)  and 
i„^k,  and  taking  0j=0  (io,ko,0o)  we  have  \0  (i;0o.ko)— 0.|  =  |®  (i:0i.i<>)— #*|  <  or  for  all  i^ij. 

□ 

Our  proofs  of  Theorem  3.5  and  Corollary  3.5  use  only  the  definition  of  uniform  asymptotic 
stability.  (Asymptotic  stability  of  9.  as  a  solution  of  the  ODE  (S.108)  is  uniform  by  virtue  of  the 
fact  that  (3.108)  is  time-invariant.)  In  an  application  of  these  results  any  demonstration  of  the 
asymptotic  stability  of  0.  is  sufficient.  Two  of  the  most  commonly  used  methods  are  verification 
via  simulation  or  an  application  of  Lasalle's  theorem.  Another  approach  to  obtaining  information 
about  0(k:0„.ko)  that  is  valid  for  all  k^k„  is  to  first  find  a  Lyapunov  function  which  proves  the 
asymptotic  or  exponential  stability  of  a  solution  or  an  invariant  set  ol  he  ODE  (3.108)  and. 
second,  make  use  of  this  Lyapunov  function  in  a  study  of  (3.104)  or  (3.136).  This  approach  does 

i  _  i 

not  explicitly  relate  0  or  0  toU.  but  instead,  provides  information  about  the  behavior  of  0  or  0 
relative  to  the  asymptotically  stable  solution  or  set.  In  the  case  of  an  asymptotically  stable 
equilibrium  arguments  similar  to  the  proofs  of  Theorem  3.5  or  Corollary  3.5  can  be  applied  to 
obtain  the  infinite  time  approximation  results  such  as  (3.145)  or  (3.154)-(3.155).  In  the  case  of  an 
invariant  set.  the  best  one  can  hope  for  is  to  establish  the  existence  of  an  invariant  set  for  (3.104) 
or  (3.136)  and  to  apply  Theorem  3.4  or  Corollary  3.4  over  finite  time  intervals.  For  simplicity  and 
because  it  fits  an  application  in  Chapter  4.  we  illustrate  this  approach  for  the  constant  gain  case 
under  the  following  assumption. 

Assumption  3.5:  There  exist  scalars  c(l>0.  y,>>0.  and  -yq  6(0. c, ,y,.)  and  a  vector  0.60  such  that 


B(c„.0.)C© 


(3.156) 
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(0-«.)Tf(0)+TT(0X0-e.)  <  2[-y„|0-0.|2+y,|0-0.|] 


(3.157) 


for  all  0€B(co.0.). 


Theorem  3.6:  Suppose  that  Assumptions  3.4  and  3.5  hold.  Given  any  (^(O.Co- — ).  there  exists 

yo 

€7(or)€(0.€4]  such  that  for  each  e6(0.€7),  any  k06Z  and  any  0o€B(co— eV:Pf,0.) 

|0(k;0o>ko)-0.|  <  (l-€yo)lt'k°(|0o-«‘l+^PF)+[l-(l-€yo)1C"koK— +o-)  (3.158) 

Vo 

for  all  k  ^k0- 


Proof:  Choose  e7  so  that 


0<€7<€4  .  — +cr<c((— ej'Pf  .  —(.K(ei')+€rplpf+e1p()+ei/l(pr/2)^cr.  (3.159) 

y0  y<> 

Let  \  Suppose  that  0(k:cr(),k(>)€B(co“< e‘'pf.Q.)  for  all  k€[k0.i].  Then  0(k.0o.ko)€0  f°r 

k€[k„.i+.\]  which  implies  that  0(k;0„.ko)  is  defined  for  k€[k„.i+l].  Using  (3.157)  and  Lemma  (3.8) 
we  have  for  all  k€[k„.i] 


|0(k  +  l;0o.k„)-0.|  <  ( 1  — €y„)|0( k ;0o.k„)— 0.|  +  |f2(k;0o.ko.e)| 


(3.160) 


from  vcnich  it  follows  that  (3.158)  holds  for  all  k£[k0.i+l]-  But  this  implies  that 
0( k.0,,.k,,)€B(c, ,— €  pF,0.)  for  all  k€[k„.i+l].  Thus,  if  0(k:9o.ko)6B(co-e:,:pF.0.)  for  all  k€[k„.i]  and 
an  arbitrary  i.  then  0(k;0,,.kt,)6B(c„— €':pF,0.)  for  all  k€[k0.i  +  l].  Since  0,,€B(c„— €'!pF.0.).  it  follows 
that  0( k;0,,.k,,)€B( c,, — e':pF.0»)  for  all  k^k(>.  Hence  (3.160)  and  (3.158)  hold  for  all  k^k„. 


It  is  clear  that  this  result  can  be  combined  with  Theorem  3.3  to  obtain  results  sii  ilar  to 
Corollaries  3.2  and  3.3  for  the  svstem  (3.1)-(3.2).  It  is  also  obvious  that  we  could  postulate  many 
different  assumptions  about  the  behavior  of"5(r;0n)  or  about  f(0).  and  then,  using  Lemmas  3. 6-3. S 

derive  results  that  applv  to  0(k:0,,.k,,)  or  0  ( k;0,,.k.F).  However,  we  feel  that  Theorems  3. 4-3. 6  and 
Corollaries  3. 4-3. 5  proviue  sufficient  illustration  of  the  use  ot  Lemmas  3. 6-3. 8  lor  the  reader  10  be 


on 
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able  to  state  and  prove  results  which  are  applicable  in  each  different  situation. 

W  e  now  consider  the  problem  of  establishing  a  connection  between  solutions  of  the  ODE 
(3.108)  and  the  system  (3.104)  or  (3.136)  when  the  input  to  the  system  is  a  sample  path  of  a 
stochastic  process.  Since  every  sample  path  of  the  input  process  is  a  deterministic  time  sequence, 
we  can  check  whether  Assumptions  3. 1-3.4  are  satisfied  on  a  sample  path  by  sample  path  basis. 
For  each  sample  path  for  which  the  assumptions  are  satisfied  Theorems  3.4  and  3.5  hold.  Hence,  if 
we  place  conditions  on  the  stochastic  process  which  generates  the  input  such  that  Assumptions 
3. 1-3. 4  are  satisfied  for  almost  every  sample  path,  then  Theorems  3.4  and  3.5  hold  with  probability 
one  (w.p.l).  If.  in  addition.  Assumption  3.5  is  satisfied  w.p.l,  then  Theorem  3.6  holds  w.p.l. 

Recall  from  the  previous  sections  that  the  only  property  of  the  input  that  is  used  in 
Assumptions  3. 1-3. 3  is  uniform  boundedness.  Therefore,  we  shall  require  almost  every  sample 
path  to  be  a  uniformly  bounded  sequence.  The  supremum  over  k€Z  can  depend  on  the  sample 
path.  However,  there  <'hould  exist  a  single  bound  which  holds  for  almost  every  sample  path.  It  is 
easier  to  give  sufficient  conditions  for  Assumption  3.4  to  be  satisfied  along  any  particular  sample 
path  than  to  say  what  conditions  are  necessary  for  Assumption  3.4  to  be  satisfied.  Each  sample 
path  could,  for  example,  be  the  sum  of  a  finite  number  of  sinusoids  with  different  sample  paths 
having  different  magnitudes,  phases,  frequencies,  or  numbers  of  sinusoids.  The  lack  of  dependence 
of  f  on  k  is  most  easily  met  by  restricting  the  input  process  to  be  a  stationary  stochastic  process. 
This  is.  in  fact,  a  very  natural  restriction  given  that  our  goal  is  to  reduce  the  study  of  (3.104)  to 
the  study  of  a  time-invariant  system.  Since  f  is  defined  as  a  time  average  after  a  sample  path  has 
been  chosen  it  can  depend  on  the  sample  path.  If  f  does  depend  on  the  sample  path,  then  we  have 
gamed  little  or  nothing  by  considering  the  input  to  be  a  sample  path  ol  a  stochastic  process.  This 
claim  follows  from  the  fact  that  we  then  must  study  the  ODE  for  each  possible  I  in  order  to  have  a 
complete  analysis,  that  is.  we  must  make  a  series  of  studies  for  different  deterministic  inputs.  I  he 
easiest  way  to  avoid  this  complication  is  to  restrict  the  input  process  to  be  ergodic.  In  this  case  the 
time  average  f ( 0 )  of  f(k,0:O)  is  equal  to  the  ensemble  average  of  f(k.0:O).  that  is.  the  expected  value 


l‘tA-4 


of  f(k,0;O).  We  summarize  this  discussion  with  the  following  lemma. 


Lemma  3.9s  Lei  the  input  to  the  system  (3.1)-(3.2),  that  is.  w(k)  and  the  k  dependence  of 

f(k  be  a  sample  path  of  a  stationary  ergodic  stochastic  process.  If  Assumptions  3. 1-3.3  hold 

for  almost  everv  sample  path,  then  Assumption  3.4  holds  with  probability  1  and  f(0)  =  E[f(k.0;O)]. 

□ 

Remark  3.11s  An  interesting  special  case  is  when  f(k.0:e)  is  linear  in  0  .  f(k.0:e)  =  0,  and 

Assumptions  3.4  and  3.5  hold  with  probability  one  for  0*  =  0  .  yj  =  0.  Theorem  3.6  then 

guarantees  exponential  convergence  to  an  arbitrary  small  ball  around  the  origin  with  probability 

one  This  is  related  to  the  results  of  Bitmead  and  Anderson  (1980a. b)  and  Shi  and  Kozin  (1986). 

□ 

In  many  adaptive  systems  the  function  f(k.0.x)  in  the  parameter  update  (3.2)  has  the  form 


f(k.0.x)  =  f„(0)+f1(0)col(w(k)wT(k))+f2(0)col(xxT) 
+  f3(0)col(xwT(k))+f4(0)w(k)+f5(0)x 

With  this  form  it  follows  that  under  the  conditions  of  Lemma  3.9 


(3.161) 


f(0)  =  fo(0)+f,(0)col(Rw(O))+f2(0)col(R„(O.0) 

+  f3(0)col(R„w(O.0))+f4(0)Etw(k))+f5(0)E[j/(k.0)]  . 


(3.162) 


where  Rw.  R„.  and  R,,u  are  the  autocorrelation  of  w  .  the  autocorrelation  of  v  and  the 
crosscorrelation  of  v  and  w.  respectively.  Because  ^(k.0)  is  the  output  of  a  linear  time-invariant 
system  with  stationary  input  w(k).  we  can  compute  Rj/O.O)  and  R„W(O,0)  via  Parseval's  theorem 
using  the  power  spectral  density  of  w  and  the  transfer  function  from  w  to  v.  Hence,  we  can  use 
the  Theorems  3. 3-3. 6  and  Lemma  3.9  to  analyze  the  efTect  of  the  frequency  content  of  w(k) 
relatic e  to  the  transfer  function  of  the  system  (3.1)  on  the  behavior  of  the  system  (3.1 )-( 3.2 ). 
This  is  discussed  in  more  detail  in  Chapter  4. 


3.6.  Concluding  Remarks 

Following  continuous-time  proofs,  we  have  established  conditions  for  the  existence  of  an 
exponentially  attractive  integral  manifold  for  slow  adaptation  in  discrete  time.  We  have  also  given 
proofs  of  averaging  theorems  for  the  analysis  of  the  on-manifold  behavior  of  slowly  adapting 
systems  with  deterministic  inputs.  Finally,  we  have  discussed  the  relationship  between  the 
deterministic  averaging  results  and  the  ODE  method  for  the  analysis  of  stochastic  adaptive 
systems. 


CHAPTER  4 


REDUCED-ORDER  MODEL  REFERENCE  ADAPTIVE  CONTROL 

4.1.  Introduction 

In  Chapter  2  we  established  the  existence  of  an  integral  manifold  for  a  standard  model 
reference  adaptive  control  system,  namely,  the  Narendra.  Valavani  (1978)  controller  for  relative 
degree  one.  Then  using  the  method  of  averaging  we  analyzed  the  behavior  of  the  adaptive  system 
when  this  controller  is  applied  to  a  plant  that  does  not  satisfy  the  exact  matching  and  SPR 
assumptions  under  which  the  controller  was  designed.  For  slow  adaptation,  we  showed  that  the 
exact  matching  and  SPR  assumptions  can  be  replaced  by  approximate  matching,  that  is.  small  RMS 
error  and  "signal  dependent  SPR"  assumptions. 

This  result,  by  itself,  gives  us  the  ability  to  design  reduced-order  model  reference  adaptive 
control  systems  because  we  can  design  the  usual  full-order  controller  for  an  assumed  plant  of 
lower  dimension  then  the  actual  plant.  However,  such  an  approach  suffers  from  the  inadequacies 
of  the  usual  full-order  controller  design.  First,  the  number  of  adjustable  parameters  is  determined 
by  the  assumed  order  of  the  plant  and  not  by  the  number  of  adjustable  parameters  which  the 
controller  needs  to  achieve  acceptable  performance.  Second,  the  usual  procedures  assume  only  that 
the  plant  is  a  black  box  of  known  order,  hence,  do  not  take  advantage  of  much  information  which 
is  usually  available  about  the  plant.  Clearly,  the  two  problems  are  related.  By  assuming  that  so 
little  information  is  available  about  the  plant  and  by  making  exact  matching  the  only  acceptable 
performance,  the  design  is  forced  to  include  as  many  adjustable  parameters  as  required  by  the 
assumed  order  of  the  plant 

In  this  chapter  we  present  an  alternative  parameterization  of  the  adjustable  controller  which 
separates  the  dynamic  order  of  the  controller  and  plant  from  the  number  of  adjustable  parameters. 
This  provides  the  freedom  to  design  a  model  reference  adaptive  controller  with  mans  fewer 
ad)ustable  parameters  than  in  the  conventional  design.  The  analysis  then  proceeds  under 
assumptions  w  hich,  in  general,  can  be  verified  only  in  the  analysis,  simulation,  and  testing  phases 
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of  a  control  system  design.  The  use  of  these  assumptions  provides  our  method  with  a  very  natural 
way  to  make  use  of  information  which  is  available  prior  to  the  commissioning  of  a  control  system. 
The  analysis  is  carried  out  in  several  parts.  We  first  establish  the  existence  of  an  attractive  integral 
manifold.  Then,  sufficient  conditions  for  stability  are  derived  using  the  averaging  theorems  of 
Chapter  3.  We  conclude  this  chapter  with  frequency  domain  interpretations  of  the  stability 
conditions. 


4.2.  A  Reduced-order  Controller  Parametrization 

Earlier  adaptive  control  schemes  adjusted  as  many  parameters  as  required  by  the  assumed 
order  of  the  plant.  This  choice  was  motivated  by  the  desire  of  perfect  matching  in  the 
disturbance-free  case-  However,  even  if  the  plant  order  were  exactly  known,  the  adjustment  of 
more  than  a  few  of  the  most  important  parameters  creates  difficulties,  especially  when  the  inputs 
are  not  persistently  exciting.  We  introduce  a  controller  parametrization  which  permits  a  reduced 
number  of  adjustable  parameters.  One  adjusta'  e  gain  is  assigned  to  each  element  in  the  vectors  of 
transfer  functions  Ft  and  F2  and  to  the  input  r  as  shown  in  Fig.  4.1.  The  state  representation  of  this 
parametrization  with  adjustable  parameter  vector  9  =  [j30,/3T.aT]T  is  given  by 

x0(k+l)  =  A<,x0(k)  +  b„0T0(k) 

X[(k+1 )  =  biC0x<)(k)  +  A]Xi(k)  +  bid„0T0(k) 

(4.1; 

x2(k+l)  =  A2x2(k)  +  bjCpXpfk)  +  b2n0(k) 

xfl(k  +  l)  =  bpc0x0(k)  +  ApXp(k)  +  bpdo0T<£(k)  +  bpn,(k) 

where  x,,.  x,.  x2.  and  xp  are  the  slates  of  F„.  F],  F2.  and  W.  respectively,  where  the  regressor  vector 
<£  =  [ r .cf> 1  1  ,d>;  T] r  is  given  by 


(f><  k  )  =  — C,x,(k) 

— Cix->(  k  ;-d-,v ,( k  ) 


/„(k)  =  C  x  (k)+n„(k) 


and  where  r<  k  •  is  the  reference  input  and  n,(  k ).  nj  k )  aie  disturbances.  We  have  included  an  input 
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n,(k)  n0(k) 


Fig.  4.1.  Reduced-order  parametrization  with  one  gain  per  transfer  function. 


disturbance  and  an  output  disturbance  with  the  idea  that  the  input  disturbance  should  represent 
inputs  to  the  plant  such  as  load  variations  and  the  output  disturbance  should  represent,  for 
example,  measurement  noise.  The  only  output  to  which  we  refer  is  y0  representing  the  measured 
output.  This  must  be  taken  into  account  when  specifying  the  desired  performance  or  evaluating 
the  actual  performance  of  the  system.  The  compensator  transfer  functions  F,,  .  F,  ,  and  F2  and  the 
plant  transfer  function  W  are  related  to  (4.1)-(4.2)  by 


F„(z)  =  c,,(zl— Ao)  lb„  +  d0  . 
F2(z)  =  C2(zl— A2)-lb2  +  d2  . 


F,(z)  =  C,(zl— A[)-1b, 
Wfz)  =  cp(zl— A,,)-1bp  . 


(4.3) 


The  number  of  adjustable  parameters  is  determined  by  the  number  of  compensator  outputs,  that  is. 
n..  =  m  +  n+1  where  Cj  has  m  rows  and  (A  has  n  rows  The  dynamic  order  of  the  compensators  F,,. 
1  , .  p2  is  at  the  designer's  disposal  and  the  number  of  parameters  is  not  dependem  on  the  order  ol 
the  compensators.  While  we  havt.  combined  all  the  states  ol  F,  and  F2.  respectively,  into  \ !  and  x2 
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for  a  convenient  state  representation,  we  do  not  imply  by  (4.3)  that  the  poles  of  each  transfer 
function  element  of  Fj  or  F2  are  the  same.  That  is.  Cj  .  Aj  .  C2  .  andA2  can  be  block  diagonal. 
However,  we  note  that  if 


F0(z)  =  1  .  F,(z) 


F2(z)  = 


(4.4) 


where  Zm(z)  has  order  then  this  parametrization  is  equivalent  to  the  full-order 

parametrization  normally  used  in  the  design  of  adaptive  controllers  based  on  the  black  box 
assumption.  Hence,  our  parametrization.  which  allows  a  reduced  number  of  parameters,  is  more 
general,  not  less  general,  than  the  controller  parametrizations  usually  encountered  in  the  adaptive 
control  literature. 


Taking  advantage  of  the  freedom  offered  by  this  parametrization  to  work  with  a  reduced 
number  of  parameters  precludes,  in  general,  the  possibility  of  exact  transfer  function  matching  via 
the  Bezout  identity.  We  replace  the  goal  of  Bezout  matching  with  the  goal  of  minimizing  the  mean 
squared  filtered  tracking  error  between  the  reference  model  output  vm(k)  and  the  plant  output 
y\( k )  with  the  parameter  held  constant.  We  let  the  relerence  model  transfer  function  and  its 
output  be 

w„,(z)  =  cm(zl-Anl)_1bm  .  (4.5) 


n(k)  =  £  cmAj 


and  we  define  the  filtered  tracking  error  e(k)  by 


xf(k+l )  =  Afxf(k)  +  bf(v„(k)  —  yin(k)) 


e(k )  =  cfxf(k )  .  F(z)  =  cf(zl— Af )  1  bf 


l  etting  e(k.O)  denote  e(k)  when  the  adjustable  parameter  is  nelJ  constant  at  0  and  the  resulting 
lineal  time-invariant  svstem  ( 4.1  ).l  4  7)  iy  initialized  with  zero  initial  conditions  at  k  =  — oo.  th,,:  is. 


letting  elk. 9)  be  the  steadv-state  response  of  elk)  when  the  parameter  is  constant  at  9.  we  make  the 
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following  assumption. 


Assumption  4.1:  For  the  uniformly  bounded  signals  r(k),  n^k) .  n0(k)  entering  the  system  (4.1) 
and  for  the  reference  model  Wm(z).  there  exists  0*  which  provides  an  isolated  local  minimum  of  the 


RMS  tracking  error 


E(0)  &  Uvgle2(-.0)]r 


A  1  k+N-1 

=  Him  -L  Z  e2(i .0)}* 

N— <ao  n 


where  the  limit  exists  uniformly  in  k  €  Z. 


Remark  4.1:  Assumption  4.1  implies  that  the  linear  time-invariant  system  (4.1)  with  constant  0  is 
exponentially  stable  at  0=0*  and  in  an  open  neighborhood  around  0*. 


This  assumption  requires  the  use  of  a  priori  knowledge  about  the  plant  or  range  of  possible 
plants  W(z).  However,  because  the  assumption  is  made  for  the  linear  time-invariant  system  (4.1) 


with  constant  0.  it  requires  essentially  the  same  information  that  is  necessary  to  design  a  fixed 


parameter  controller  with  this  structure.  First  the  compensators  F()  .  F!  .  and  F2  must  be  chosen  so 
that  each  fixed  plant  in  the  range  of  possible  plants  can  be  stabilized  for  some  value  of  0.  Then, 
taking  into  account  the  expected  input  signals,  or  designing  the  input  signal,  a  reference  model  and 
an  error  filter  are  chosen  which  reflect  an  estimate  of  the  achievable  performance.  That  is.  the 
reference  model  and  error  filter  should  be  chosen  so  that  the  RMS  error  E(0)  can  be  made  small. 
The  advantages  of  a  small  E(0*)  become  clear  in  the  sequel.  While  the  satisfaction  of  these 
requirements  may  imply  a  significant  off-line  design  effort,  this  effort  is  justified  by  the  improved 
robustness  in  the  on-line  adaptation. 

Remark  4.2:  Small  E(0’)  imply  transfer  function  matching.  It  only  requires  that  the  transfer 
functions  be  close  at  the  dominant  frequencies  of  the  inputs  to  the  system. 


Although  the  number  of  parameters  has  been  reduced,  the  structure  of  the  proposed  controller 
preserves  the  appearance  of  the  parameter  vector  0  in  (4.1)  in  the  familiar  parameter-regressor 
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product  9T<f>.  Letting  yo(k.0) .  <f>(k ,0)  denote  the  steady-state  of  y0(k)  .  <f>( k).  respectively,  with  the 
parameter  vector  held  constant  at  0,  we  note  that  the  system  (4.1)  has  the  property 

yo(k,0)  -  yo(k.0‘)  =  WCL(0,.z)[0T(k.e)(0-0*)]  (4.9) 

with  /3OWCL(0,2)  being  the  transfer  function  from  r  to  y0 

^  F0(z)W(z) 

WCL(0,z)  =  - - -  ,  (A  10) 

1  +  F0(z)(/3TFj(z)  +  aTF2(z)W(z)) 

and  where,  by  the  mixed  k.z  notation  in  (4.9)  we  mean  that  yo(k.0)— yo(k.0‘)  is  the  steady-  state 
output  of  the  transfer  function  WCL(0\z)  with  input  #T(k,0)(0— 0*). 


4.3.  Parameter  Update  Law 

We  denote  the  system  matrix  of  (4.1)  by 

A(0)  =  A  -  b0TC 

where  the  constant  matrices  A .  b,  and  C  are 


(4.11) 


A«  o 

0 

0 

b,» 

bic0  A, 

0 

0 

u  — 

bid0 

0  0 

a2 

b2Cp 

,  D  — 

0 

bpC0  0 

0 

Ap 

bpdo 

(4.12) 


C  = 


0  0  0  0 
0  Cl  0  0 

o  o  c2  d2Cp 


From  Remark  4.1  it  follows  that  Assumption  4.1  implies  Assumption  3.1  holds  with  A (0)  replaced 
by  A(0).  that  is.  there  exist  a  compact  set  0  containing  0’  and  an  open  neighborhood  of  O'  and 
constants  A, ,  €  (0.1)  and  K  ^  1  such  that 


V,  A 
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1 


,V\ 

V1 


£ 


I 


$ 


i 


.3 


I 


M 


I 


I  A‘(0)  I  <  KX<i  Vi  ^  0  ¥0  €  0  .  (4.13) 

We  assume  that  our  a  priori  knowledge  includes  at  least  one  point  in  the  set  9  at  which  our 
parameter  vector  is  initialized.  Then,  the  task  of  parameter  adaptation  is  to  improve  performance 
and  track  slow  changes  in  the  plant.  Our  use  of  slow  adaptation  has  two  advantages: 

(i)  the  inherent  stability  of  the  fixed  parameter  controller  quantified  by  (4.13)  is  preserved  for 
slow  variations  of  the  plant  parameters  which  otherwise  could  cause  instability. 

(ii)  the  parameters  to  not  overreact  to  the  misinformation  that  accompanies  a  nonzero  minimum 
of  the  RMS  tracking  error. 

The  parameter  9  is  updated  at  every  instant  k  by  a  small  step  which  is  proportional  to  the 
product  of  the  filtered  regressor  vector  £ 

£‘(k+l)  =  Amf£‘(k)  +  bm{0j(k) . 

£,(k)  =  cmf|‘(k)  .  i=l,2 . ne. 

F(z)Wm(z)  =  cmf(zl— Amf)-1  bmf  . 

the  filtered  tracking  error  e(k),  and  the  step  size  € 

0(k+l)  =  0(k)— e£(k)e(k) . 

The  choice  of  a  constant  filter  to  get  t,  from  0  and  the  choice  of  F(z)Wm(z)  as  this  constant  filter 
have  special  significance  for  slow  adaptation  with  the  reduced  parameterization  (4.1).  The 
motivation  for  this  choice  goes  back  to  the  method  of  sensitivity  points.  (Kokotovic.  1973).  By  this 
method  the  gradient  of  the  output  yo(k,0)  with  respect  to  the  constant  parameter  0  is  obtained  by 
passing  0  through  the  error  filter  F(z)  and  the  exact  closed-loop  system  transfer  function  WCL(0,z). 
Fig.  4.2.  In  particular,  this  holds  at  0’  with  0’  and  WCL(0’,z).  At  O',  we  obtain  £  bv  passing  0’ 
through  F(z)Wm(z).  If  the  part  of  the  filtered  output  F(z)[vo(k,0)]  due  to  n,  and  n0  is  small  relative 
to  the  part  due  to  r.  then  by  the  definition  of  9 '  .  Win(z)  is  near  the  best  RMS  approximation  of 
0,;wtL(0\z):  hence.  £  is  proportional  to  a  good  approximation  of  the  gradient.  For  this  reason  the 
filtered  regressor  £  is  also  called  the  "pseudogradiem"  (Kokotovic.  Medanic.  Vuskovic.  and  Bingulac. 


(4.14) 


(4.15) 


Fig.  4.2.  Sensitivity  points  for  obtaining  the  gradient  of  the  output. 


1966).  This  gradient  approximation  property  will  be  used  to  show  that  the  parameters  converge  to 
a  neighborhood  around  0*  with  radius  proportional  to  E2(0*).  Using  the  property  (4.9)  of  the 
system  (4.1)  we  see  that  the  steady-state  response  of  the  (J(k)e(k)  with  constant  parameter  6. 
denoted  by  £(k,0)e(k.0),  is  given  by 


<(k.0)e(k.0)  =  [F(z)Wm(z)  0(k.0)]  [I  (z)\V(  L(0’,z)</>(k.0)]T  (0-0')  +  e(k.0‘)  .  (4.16) 


Notice  that  the  first  term  on  the  right  side  of  (4.16)  is  the  product  of  a  time-varying.  0-dependent 
matrix  and  the  parameter  error  0—0’.  This  structure  is  used  to  develop  estimates  of  the  region  of 
attraction  of  an  exponentially  stable  invariant  set  containing  O'. 


y.y. 


Letting  XT  =  [xT,xfT.|T]  where  xT  =  [xo^x/.x/.x*]  and  |T  =  [£1T.£2T . and  letting 

w(k)  =  [r(k).n,(k).np(k).ym(k)],  we  write  the  system  (4.1),  (4.7),  (4.14),  (4.15)  in  the  form  of 
(3.1M3.2). 


where 


X(k+1)  =  A(0(k))X(k)  +  B(0(k))w(k; 
0(k+l)  =  0(k)  +  ef(X(k)). 


A(0) 

0  0 

Bt(0) 

A(0)  = 

A2l 

Af  0 

B(0)  = 

B, 

A31 

r 

r*7 

CO 

< 

o 

1 

b3 

B,(0)  = 


P<t>b<t>  0  aTd2b<>  0 
0*b,d*  0  atTd2bjd*  o 
0  0  b2  0 

^b^d^  bp  a T(i2bpd0  ® 
f(X)  =  -CmX  ceX 


(4.17) 

(4.18) 


(4.19) 


with  A21  .  Af  .  B2  .  and  c,.  being  the  constant  matrices  corresponding  to  the  tracking  error  filtering 
(4.7)  and  with  A31  .  A33  .  B3  .  and  Cm  being  the  constant  matrices  corresponding  to  the  regressor 
vector  filtering  (4.14).  The  block  triangular  structure  of  A(0)  and  (4.13)  imply  that  Assumption 
3.1  holds.  Defining  the  frozen  parameter  response 

Kk.0)  =  £  Ak-‘-'(0)B(0)w(i)  (4.20) 

l  =  —rx> 

it  follows  from  the  boundedness  of  w.  the  stability  of  A(0).  and  the  linear  dependences  of  A  and  B 
on  0.  that  Assumption  3.2  holds.  Finally,  we  note  that  f(X)  is  quadratic  in  \:  hence.  Assumption 
3.3  holds  and  Theorem  3.3  guarantees  the  existence  of  a  local  integral  manifold  of  (4.17 )-( 4 . 1 S ). 
Theorem  4.1:  Suppose  that  Assumption  4.1  holds.  Then,  the  system  (4.17 )-( 4.18)  satisfies 
Assumptions  3. 1-3.3:  hence,  for  any  given  Do  >  0  .  A,,  >  0  ,  D,  >  D,,  .  and  X  €  (\,,.l)  there  exists 
€4(D,,.A,..D|.X )  >  O  such  that  for  each  e  €  [(),€4)  there  exists  a 


f  unction 


h(k.0:e)  €  H((€/€4)D0.(e/€4)A0)  with  the  following  properties.  Let  X(k),0(k)  be  the  solution  of 
(4.17)-(4.18)  with  initial  data  X(k0)  =  X0 . 0(ko)  =  0O  and  let  g(k.0:e)  =  i/(k,0)  +  h(k.0;€).  If 
0(k)  €  ©  for  all  k  €  [ko.kj,  then 

(i)  X0  =  g(ko,0o:e)  implies  X(k)  =  g(k,0(k);e)  ¥k  6  [ko.kj 

(ii)  IX0  —  i/(ko.0o)  I  ^  Di/K  implies  that  Vk  6  [k0.kj 

I  X(k)  -  g(k.0(k);e  i  ^  K  \k_lc°  I  X0  -  g(ko.0o;e)  I  .  (4.21) 

□ 

Remark  4.3:  If  the  vector  of  input  signals  w(k)  is  N-periodic.  then  g(k,0:e)  is  N-periodic  in  k. 

□ 

Remark  4.4:  In  Theorem  4.1  we  have  not  used  in  any  essential  way  the  fact  that  0’  provides  a 
minimum  of  E(0)  or  that  the  limit  in  the  definition  of  E(0)  exists  uniformly  with  respect  to  k.  We 
have  used  only  the  boundedness  of  w(k)  and  the  implied  stability  of  A(0). 

□ 


4.4.  Stability  in  the  Slow  Manifold:  Averaging 

The  adaptive  system  (4.17 )-( 4. IS)  restricted  to  the  slow  manifold  Me  =  |k.0.X  :  X  =  g(k.0;€)l 
evolves  according  to  X(k)  =  g(k,0(k);e)  and 


0(k+l)  =  0(k)  +  ef(g(k.0:€)) . 


(4.22) 


We  apply  the  results  of  Section  3.5  to  obtain  sufficient  conditions  for  (4.22)  to  possess  an 
exponentially  stable  invariant  set.  We  define  the  averaged  system  or  ODE 


where 


Af=M) 

dr 

1(0)  —  avg[f(i/(-.0))] 

=  —  avg[£(  ,0)e(  -  .0 )] 


(4.23) 


(4.24) 


with  cf>(  k.0 )  ,  ^( k .0)  .  e(  k.0 )  denoting,  respectively,  the  steady -state  response  of  0(k )  .  £( k ).  e(  k ). 
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that  is. 


0(k,0)  =  -  0  £(k,0)  =  Cm  Kk .0)  .  e(k.0)  =  ce  Kk.0)  .  (4.25) 

Cv(k.B) 

d2n0(k) 

In  order  to  take  advantage  of  the  structure  implied  by  (4.9).  we  introduce  v(k.0.0‘).  the  frozen 
parameter  regressor  vector  <f>( k,0)  filtered  by  F(z)WCL(0*.z). 


v(k,0.0')  =  F(z)WCL(0'.z)0(k.0)  . 


(4.26) 


Using  (4.9)  and  (4.25)  we  write  f(g(k.0;O))  as 


f(v(k,0))  =  -{(k^v^kJ.rXO-T)  -  £(k.0)e(k,0') 


(4.27) 


Defining 


R(0.0')  =  a  vg[£( •  .0 ) vT(  .0.0’ )] 


(4.28) 


b(0.0‘)  =  avg[{(-.0)e(-.0')] 


(4.29) 


we  rewrite  f  as 


f(0)  =  — R(0,0*)(0— 0’)  -  b(0.0')  . 


(4.30) 


From  (4.30)  it  is  clear  that  the  averaged  system  (4.23)  has  the  same  structure  as  the  averaged 
system  which  was  studied  in  Section  2.6.  Hence,  we  could  develop  the  parallel  results  for  the 
discrete-time  adaptive  system  (4.17 )-( 4.18)  using  Theorems  3.4  and  3.5.  However,  we  choose  to 
obtain  our  sufficient  conditions  for  stability  with  the  more  direct  approach  of  Theorem  3.6.  Letting 


y/Kj.O*)  =  max  III  £( -.0)e(\0  ) 


(4.31) 


where  II  III  denotes  the  RMS  value  of  the  Euclidean  norm. 


I  <(-.0)HI  =  favg[|<(  ,0)|-]} 


(4.32) 


we  note  that 


wm 


|b(0.0  )|  <  )  ¥06B(Ki.0*)  . 

Theorem  4.2:  Suppose  that  Assumption  4.1  holds,  that  B(Ki.0*)£6.  and  that 

R(0,0*)+RT(0,0*)  >  2yoI>0  ¥06B(Ki.0*)  . 


Given  any  D0>O.A0>O.D1>D0,  and  X6(X0.l).  if 


Vi(Kj.0  )  <  Kjyo 


(4.33) 


(4.34) 


(4.35) 


then  for  any  O’€(0.K1  —  — )  there  exists  €*(oO€(0.€4(D0.A0,D1,\)]  such  that  for  each  €€(0.€»).  any 

Vo 

k0€Z.  anv  0o€B(K1— €V;pf(Dq)— e-P-^.Dl)  -0*)  and  any  6 B( D j/K ,v( ko.do))  the  solution  X(k), 

1  A 

0(k)  of  (4.17)-(4.18)  with  initial  data  X(k0)  =  X0.  0(ko)  =  0O  satisfies  (4.21)  and 


|0(k)-0*|  <  (l-€y0)k'ko  l0o-9*l+eV!pF+eEl^Ell  - 


—+<r  +  (—+<r)  (4.36) 


for  all  k^k0. 


Proof:  In  the  proof  of  Lemma  3.6  we  replace  f(k.0;e)  by  f(X(k))  and  compute  the  new  bound 


|0(k)— 0(k)|  <  *  Z  |0(k+i)— 0(k)|  <  -I  N£  ZiePf+ep^DOD^^] 

^  1=0  ^  i=()  i=0 


v  \*— 

^  i=0  |j=() 


1  N-1  D,p,(D,) 


^  (eN— e)(pF/2)+e 


D1pz(D1) 


In  the  proof  of  Lemma  3.8  we  redefine  f;>4  as 


f24(k:X().0o.k„.e'i  =  -i-  Z  €[f(X(k)-r(v(k.0(k)))]  . 

'  l=«* 


and  similarly  to  (4.37),  we  replace  the  bound  (3.128)  by 


lr  (\  v  n  *  .j  ->  ,  €  OiP7(I)i) 

|f24(k;X(„0„.k„.e)|  ^  e~pf  +  — -  — - — 

.\  1  — A 


97 


Noting  that  Lemma  3.6  was  used  to  obtain  (3.126)  we  replace  it  by 

P22I  ^  e(€N— e)(piPf/2)  +  e- - — - - 

The  proof  is  completed  by  following  the  proof  of  Theorem  3.6  with  (3.160)  replaced  by 

>1  ,  <  v,  DiPz(D,) 

—  +o-  <  k,-ev-pF-€ — — — 

Vo  IX 

—  K(€V:)+€V:plpF+epe+(€V:+ep1)^i^£ —  +  ev'ipf/2)+e <  a  . 
Vf.  1— \  1~A 


(4.40) 


(4.41) 


Corollary  4.1:  Suppose  that  w(k)  is  a  sample  path  of  a  stationary  ergodic  stochastic  process.  If 
the  hypotheses  of  Theorem  4.2  are  satisfied  by  almost  every  sample  path  of  the  process  generating 
w(k),  then  the  conclusions  of  Theorem  4.2  hold  with  probability  1. 

□ 

Corollary  4.2:  If  w(k)  is  N  periodic,  then  e'/;pF(D0)  can  be  replaced  with  e.\’pF(D0)  in 
Theorem  4.2. 


4.5.  Frequency  Domain  Interpretation  of  Theorem  4.2 

In  this  section  we  evaluate  and  interpret  the  stability  condition  (4.34).  In  the  course  of  this 
study  we  relate  the  input  signal  frequency  spectrum  to  the  stability  of  the  adaptive  system 
(4.17 )-( 4.18)  and  investigate  the  effect  of  choosing  reference  models  for  which  the  minimum  RMS 
error  E(0  )  is  small. 

For  ease  of  exposition  we  assume  temporarily  that  there  are  no  disturbances,  that  is, 
n,(  k  )  =  n  ,( k  )=(>.  In  order  to  give  an  interpretation  of  the  stability  condition  (4.34)  in  terms  of  the 
input  signal  spectrum  and  the  pertinent  transfer  functions,  we  take  r(k)  as  a  finite  sum  ol 
sinusoids. 


r(  k  )  =  £  r,„eJ','k  .  r_,„  =  r,„ 


(4.42  ) 


where  is  the  complex  conjugate  of  r^.  the  set  (1  has  a  finite  number  of  elements, 
implies  — <a€  fl.  Denoting  by  G(0,z)  the  vector  of  transfer  functions  from  r  to  <f>.  that  is. 


G(0.z)  = 


1 

l-azi-AW))-^ 


we  compute  the  Fourier  series  representation  of  <j>.  v.  and 

0(k,0)  =  22  G(0.eJ“)rueJ“*  . 

v(k.0.0*)  =  Z  G(0.eJ“)F(eJ“)WCL(0*.eJW)r<iJeJ“k 

oj€  0 

$(k.0)  =  £  G(0.eJ“)F(eju)Wm(eJ“)rweJ“k  . 

uj£  ft 


From  (4.45)  and  (4.46)  we  calculate  R(0.0  ) 


R(0,0  )  =  £  G(0.eJ“)GT(0.elj“)  |F(e>**)|2  |rJ2Wm(e^)WCL(0  .e"J“)  . 

<u€  ft 

and  restate  the  stability  condition  (4.34)  in  frequency  domain 
0  <  2ynI  <  R(0.0*)  +  RT(0.0*) 

=  22  G(0,eJw)GT(0.e-J“)  |F(e^)p  |rJ2Re(Win(eJW)WCL(0*.e-J“))  .  ¥06B(K, 

ft 

A  necessary  condition  for  (4.48)  to  be  satisfied  is  that  for  some  y>0 

0  <  yl  22  G(0,e^)GT(0.e-^)  |rj2 

=  avg[0(-,0)d>T(-.0)]  .  ¥06B(Ki.0*)  . 


hich  is  clearly  a  persistent  excitation  (PE)  condition  on  the  regressor  vector. 


When  the  minimum  RMS  error  F(0  )  is  small,  we  can  show  that  a  PE  condition  on 


and  o>€fi 

(4.43) 

(4.44) 

(4.45) 

(4.46) 

(4.47) 

,9-)<4-48) 

(4.49) 

the  filtered 


regressor  k.0 )  is  a  sufficient  condition  for  the  stability  condition  ( 4.48)  to  hold.  Petting 
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it  follows  that 


wt  =  max  III  £(•. 0 )  1 


g  =  max  max  G(0.e*O 

b  *.  <j6ft  '  1 

96B(K,.e  ) 


(4.50) 


$(-.9)-/3ov(-.e.e*)in  ^gE(e*) 


(4.51) 


where  we  have  the  clear  interpretation  of  g  as  the  gain  from  r(k)  to  0(k,0)  and  of  E(0  )  as  a 

measure  of  transfer  function  mismatch  j3o*F(z)WCL(0*.z)  —  F(z)Wm(z)  at  the  frequencies  &>€ft  of 
the  reference  input  r(k).  Using  this  bound  and  assuming  a  PE  condition  on  the  filtered  regressor, 
that  is. 


0  <  >1  <  avgU(-.e)<T(-.0)] 

=  £  G(0.eJ“)  GT(0.e"J“)  |F(eJ“)|2|Wm(e^)|2|rJ2  .  V0€B(K,.0*) 

U  u 


(4.52) 


we  have  the  following  corollary  to  Theorem  4.2. 

Corollary  4.3:  Suppose  that  Assumption  4.1  holds,  that  n^n^sO.  that  r(k)  is  given  by  (4.42), 
and  that  (4.52)  holds.  If  /30* >  0  and 


y„  =  — [y— WigE(0*)]  >  0  . 


(4.53) 


then  (4.34)  is  satisfied:  hence,  if  (4.35)  is  satisfied,  then  the  conclusions  of  Theorem  4.2  hold. 


Thus,  when  the  reference  model  and  reference  input  are  such  that  the  error  E(0)  can  be  made 
small,  the  stability  condition  (4.34)  reduces  to  a  PE  condition  (4.52)  on  the  filtered  regressor.  An 
important  point  to  remember  is  that  this  PE  condition  is  checked  pointw  ise  in  0  lor  constant  values 
ol  0€B(K,.0*).  That  is.  the  vector  £(k.0)  which  must  be  PE  is  the  output  of  a  linear  time-invariant 
s\  stem  driven  by  r(k):  hence,  the  PE  requirements  on  £  are  readih  shilled  to  sufficient  richness 
conditions  on  r.  We  see  that  r  must  contain  frequencies  lor  which  (>  E.  and  W„.  are  not  loo  small 
*  * 

and  for  which  j3  ,  WCL(0  , j<o ) —  Wm(  joi)  is  small. 


£***/*“«  *'»  V  -  X'O  -v  ^  \ 


«  “  '  V  a-  ‘  »  V  *  W>V-  V"  '»•< 


Equation  (4.48)  also  points  out  one  of  the  advantages  of  using  the  filtered  regressor  vector  £  in 
the  update  law  rather  than  the  regressor  vector  <f>.  Recall  from  Chapter  2  that  the  frequency 
domain  interpretation  of  (4.34)  was  the  "signal  dependent  SPR"  condition  on  WCL(0*  .  •).  From 
(4.48)  notice  that  with  regressor  filtering  by  Wm  we  have  a  "signal  dependent  SPR*  condition  on 
Wm( <?■*“)  WCL(0\e~J“).  For  small  E(0‘)  ,/3o'WCL(0’  ,  •)  is  close  to  Wm(z)  at  the  frequencies  co€f l  and 
the  positivity  condition  on  Wm(eJ“)WCL(0*  ,  e-J“)  is  almost  trivial.  Hence,  the  regressor  vector 
filtering  removes  the  requirement  that  our  model  Wm  be  SPR 

The  term  yj(KlP0*)  defined  in  (4.33)  is  bounded  by  yj  ^  WiE(0*)  which  implies  that  the 
radius  of  the  invariant  set  of  the  ODE  (4.23)  is  O(E(0  )).  We  use  the  fact  that  £(k.0  ) 
approximates  the  gradient  v(k.0*.0*)  of  e(k,0)  with  respect  to  0  at  0=0*  in  order  to  show  that  the 
invariant  set  actually  has  radius  of  O(E2(0*)).  Let 

w2  =  max  III  ^.£(-.0)111  .  (4.54) 

t*B(2w,E<e  )/yo,0  ) 

Clearly.  w2  is  bounded  by  a  constant  times  vj  (from  Assumption  3.2).  Letting 

g*  =  max  |G( 0* .e-10* )|  (4.55) 

idfc  u 

we  rewrite  b(0.0* )  as  the  sum  of  two  terms 

b(0.0*)  =  avg[(£(-,0)— £(-,0*))e(  -  .0*)] 

(4.56) 

+  avg[(£(-.0*)— /3o*v(-,0*.0*))e(-.0*)] 
and  bound  b (9.9*)  for  all  0£B(2w,E(0*)/y„.0*) 

|b(0.0*)|  $  g*+2-L^i  E2(0*)  .  (4.57) 

l  sing  these  bounds  we  show  that  after  converging  in  finite  time  to  the  ball  B(  2w  ,1X0  )/y,  .0*)  the 
parameters  converge  exponential!)  to  a  smaller  ball. 

Corollary  4.4:  Under  the  conditions  of  Theorem  4.2.  if  n,  =  n  =  0.  r(  k  )  is  giv  en  bv  ( 4.42  ).  and 
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v 

■*> 

i-r 


I 


0  . 

fe 


g 


+  2 


WjW2 


y0 


E(0*)  < 

Vo 


DiPf(Dj) 


<r<^E(0*)-€*-e-^ 

VI  *  i- 


(4.58) 


then 


l  0(k)  —  0*1  ^  (l-€y0)k  k'  2  -----  -  +  [1— (1— ey0)k'k’] 


Vo 


*  .  -  WjW2 

g  +2  — — 


E2(0*) 


+  cr 


(4.59) 


for  all  k  ^  kj(e)  where 


kj(e) 


1 


In 


K, 


w  jE(0*) 

Vo 


cr 


—  In 


2wjE(0  ) 


-€*(D)-e 


Dipz(Di) 

TIX 


(4.60) 

□ 


In  Chapter  2  without  regressor  filtering  the  best  we  could  hope  for  was  that  the  equilibrium 
of  the  ODE  was  O(E(0*))  from  0*.  Hence,  a  second  advantage  of  using  regressor  filtering  over  not 
using  it  is  that  it  allows  the  slowly  adapting  system  to  converge  to  a  smaller  invariant  set  around 
the  optimal  parameter  value. 

The  model  reference  adaptive  system  (4.17 )-(4. 18)  also  allows  filtering  of  the  measured  error 
between  y0  and  ym  by  the  transfer  function  F(z).  We  drop  the  assumption  of  no  disturbances  in 
order  to  investigate  the  advantages  of  this  error  filtering.  We  suppose  that  the  input  disturbance  n, 
is  a  finite  sum  of  sinusoids  with  frequencies  in  the  set  fl,. 

n,(k)  =  £  nlu>e^k  .  (4.61) 

11, 

and  that  the  output  disturbance  n0(k)  is  a  finite  sum  of  sinusoids  with  frequencies  in  the  set  fl,, 

n,.(k)=  £  ntlo,  eJwk  .  (4  62) 

We  allow  n  and  fl,  to  have  common  elements  but  we  assume  that  fl  fl  Q„  =  <b  and  fl,  Pi  fl„  =  <£. 
I.etting  G,(0.z)  and  Go(0,z)  be.  respectively,  the  transfer  functions  from  n,(k)  to  <£(k,0)  and  from 
n,,( k)  to  <fi(k.0).  we  compute  R(0,0*) 


I 


R(0.0  )  =  £  Gfd.e**)  GT(0.e~J“)  i  F(eJ“)  I  2  I  ru  1 2  Wm(e^)WCL(0*.e-J“) 


+  £  G(0.ej“)  GiT(0.e-j“)  I  F(e>")  I  2ru  n*.  Wm(eJ“)WCL(0*.e-j“) 

<j6  unit. 


+  £  Gj(0.eJU)  GT(0,e-^)  I  F(e>*)  I  2  l^n*.  Wm(eJ")WCL(0  V*) 

<j6  flOfl, 


(4.63) 


+  £  G,(0.e*O  G,T(e.e-f“)  IF(e*“)l2  I  niu  I  2  Wm(e^)WCL(0*.e-'-) 
0  = 


+  L  Go(0.e*O  GoT(0.e-JW)  I  F(e*-)  I  2  I  n^l  2  Wm(e>*)WCL(e  V>*) . 

<d€  Q 


Letting  W,(0.z)  and  Wo(0,z)  denote,  respectively,  the  transfer  function  from  n^k)  to  y0(k)  and 


from  n0(k)  to  y0(k).  we  compute  the  RMS  error  E(0) 


ECO)  =  £  I  F(eJ“)  I  2  I  ru  I  2  I  PoWCL(0.e»>)  -  Wm(e**)  I  2 


+  £  I  F(e*0  I  2  r  ji*  Wi(0.e-^)(0oWCL(e.e^)  -  Wm(e**)) 

n, nn 


+  £  IF(eJ“)l2r(11nlo>W,(0.eJ“)(0oWCL(0.e-Jw)-Wm(e-J“)) 

ii.nn 


(4.64) 


+  £  I  F(eJ<J)  I  2  I  W/d.eJ-*)  I  2  I  niw  1 2 

ufc  11, 


+  £  I  F(e>")  I  2  I  Wo(0.eJ“)  I  2  I  n^  I  2  . 


From  (4.64)  it  is  clear  that  minimizing  E(0)  requires  the  controller  to  make  I  W.ffl.e^)  I  and 


I  Wo(0.ej")  I  small  at  frequencies  oj  €  fij  U  ft0  while  also  making  I  /3oWCL(0,ej“)  —  Wm(eJ“)  I  small 


at  frequencies  w  6  ft.  If  we  further  assume  that  the  output  disturbance  contains  only  high 


frequencies  and  that  the  reference  input  and  input  disturbances  contain  only  low  frequencies,  that 


0  ^  max  I  cl»  I  <  toc  <  min  I  to  I  <  w  . 

cjfUiUU 


(4.65) 


and  that  n„(k)  is  measurement  error  which  is  to  be  ignored  rather  than  compensated,  then  we  can 


take  advantage  of  F( x )  to  make  our  cost  functional  F(0)  compatible  with  our  objective  and  to 


improve  robustness  of  the  parameter  update.  We  simply  choose  F(z)  so  that  I  Ffe^)  I  is  zero  or 


very  small  for  all  to  €  (oy.7r).  This  removes  the  effect  of  n0(k)  from  both  F(0)  and  R(0.0  ).  Notice 


that  only  the  magnitude  of  Ffe*")  appears  in  E(0)  and  R(0,0  ).  This  implies  that  we  have  no 


!r.»*V 


that  only  the  magnitude  of  FCe**)  appears  in  E(0)  and  R {9.9*).  This  implies  that  we  have  no 
constraints  on  the  phase  characteristic  of  F(e*0  when  we  design  its  magnitude  characteristic. 


Error  filtering  is,  however,  not  a  cure  for  all  problems.  If  we  want  our  controller  to  make  the 
response  of  the  plant  due  to  the  input  disturbance  n;(k)  small,  then  I  Ffe^)  I  should  be  nonzero  for 
all  to  €  ft,  so  that  our  cost  functional  reflects  our  objective.  However,  if  I  F(e*“)  I  is  not  zero  for 
each  to  €  11 4,  then  we  want  to  have  Re(Wm(ej“)  WCL(0*,e->“‘))  >  0  and  IrntW^e*1*)  W^d’.e-^)) 
small  for  all  to€  fi*  so  that  R+RT  stays  positive.  This  may  be  difficult  to  ensure  or  justify  for 
frequencies  to€  which  are  not  close  to  the  frequencies  in  ft.  Hence,  in  the  design  of  the  adaptive 
system.  I  F(ej“)  I  large  to  include  niu  in  the  cost  functional  E(0)  may  have  to  be  traded  off  against 
I  F(e]U)  I  small  to  keep  R+RT  positive.  Theorem  4.2  and  the  expressions  (4.63)  and  (4.64)  offer 
guidelines  for  this  step  in  the  design. 

Remark  4.5:  If  the  input  <o(k)  is  generated  by  a  stationary  ergodic  random  process,  then  the 
frequency  domain  interpretation  remains  valid,  but  the  sums  over  finite  sets  are  replaced  by 
integrals  over  (—• tr.7r)  and  the  Fourier  series  coefficients  are  replaced  by  the  spectral  density. 

□ 


4.6.  Concluding  Remarks 

We  have  presented  an  adaptive  control  scheme  with  a  controller  parameter  parametrization 
that  allows  for  the  design  of  model  reference  adaptive  control  systems  with  a  reduced  number  of 
parameters.  Verification  of  the  stability  conditions  for  the  parameter  update  may  require 
significant  off-line  design  effort  or  a  priori  knowledge.  This,  however,  should  be  considered  as  an 
opportunity  rather  than  a  burden  because  it  allows  the  designer  to  use  available  information  to 
reduce  the  number  of  parameters  in  the  adaptive  control  system. 
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CHAPTERS 

DESIGN  OF  SLOWLY  ADAPTING  CONTROL  SYSTEMS:  AN  EXAMPLE 
S.l.  Introduction 

Successful  application  of  the  model  reference  adaptive  control  system  introduced  in  Chapter  4 
consists  of  two  separate  developments,  both  of  which  relate  to  the  control  system  (4.1)  with 
constant  values  of  the  adjustable  gains  0.  First,  the  compensator  blocks  F0.  Fp  and  F2  must  be 
designed  so  that  the  fixed  gain  control  system  can  be  tuned  to  give  acceptable  performance  by 
adjusting  only  the  gains  0.  Since  the  main  reason  for  having  adaptive  control  is  to  combat 
parameter  uncertainty  or  variability  in  the  plant,  the  possibility  to  tune  the  control  system  by 
adjusting  only  the  gains  0  should  exist  for  the  entire  range  of  possible  plants.  The  second  step  is  to 


design  the  reference  model,  the  error  filter,  and  .if  applicable,  the  input  signals  so  that  the  9*  which 
minimizes  the  RMS  filtered  tracking  error  E (9)  provides  good  tuning  of  the  control  system.  The 
value  of  9  which  minimizes  E(0)  should  depend  on  which  plant  in  the  range  of  possible  plants  is 
used,  but  the  property  that  9*  provides  good  tuning  of  the  control  system  should  hold  for  any 
plant  in  the  range  of  possible  plants.  We  remark  that  if  0*  is  the  same  for  each  plant,  then  adaptive 
control  is  not  necessary.  Following  Kokotovic.  Medanic.  Vuskovic.  and  Bingulac  (1966).  we  shall 
say  that  a  controller  is  compatible  if  it  can  be  tuned  for  each  possible  plant  by  changing  only  0. 

These  two  steps  can  be  generalized  to  provide  guidelines  for  the  design  of  slowly  adapting 
control  systems. 

( 1 )  Given  the  range  of  possible  linear  time-invariant  plants,  choose  a  controller  parameirization 

with  adjustable  parameter  vector  0  such  that,  for  each  possible  plant. 

(a)  the  fixed  parameter  controller  is  compatible,  and 

(b)  if  the  closed-loop  system  is  written  in  the  state  space  form 


x(k+l )  =  A(0)x  +  B(0)w(k) 


(5.1) 


then  A(0)  and  B(0)  are  differentiable  with  Lipschitzian  derivatives. 
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(2)  Given  such  a  controller,  find  a  cost  functional  J(0)  such  that 

(a)  for  each  possible  plant,  the  0*  which  minimizes  J(0)  provides  acceptable  tuning  of  the  fixed 
parameter  control  system,  and 

(b)  J(0)  is  differentiable  and  its  derivative  is  Lipschitzian. 


(3)  Construct  filters  with  state  £.  inputs  x  and  w.  and  output  £.  and  construct  a  parameter 


V* 

N  ^ 


update  law 


0(k+l)  =  0(k)  +  ef(w(k).0(k),x(k).£(k)) 


so  that,  in  the  averaged  system 


A*  =7(3)  . 


CV.  .** 

/*  >\ 
yv 


g  A 

g  ' 


!« 


1 

& 


where 


f  satisfies 


7(0)  —  avg[  f(w().0.x(-.0).$(-,0))] 


7T  -  _  or  TT^_ 


Assuming  that  acceptable  performance  implies  all  eigenvalues  of  A(0  )  are  strictly  inside  the  unit 
circle,  it  follows  that  we  can  establish  the  existence  of  an  exponentially  attractive  integral 
manifold  in  a  ball  around  0*  for  the  slowly  adapting  control  system  (5.1M5.2).  In  the  manifold 
we  apply  averaging  to  investigate  the  evolution  of  the  parameters.  Assuming  isolated  local 
minima.  (5.5)  quarantees  that  solutions  of  the  the  ODE  (5.3)  beginning  close  enough  to  0  converge 
to  0  or  a  small  invariant  set  containing  9*. 

We  point  out  that  step  (1)  is  required  in  the  design  of  any  linear  time-invariant  controller 
which  is  applied  to  an  uncertain  plant,  to  a  nonlinear  plant  linearized  at  different  operating  points, 
or  to  different  copies  of  the  same  product.  Step  (2)  is  related  to  the  off-line  tuning  of  such  a 
controller.  If  the  controller  has  more  than  three  parameters,  manual  tuning  is  often  a  difficult  task. 
Automated  tuning  requires  the  specification  of  a  cost  functional  to  be  minimized.  However,  in  on¬ 
line  tuning  with  a  human  supervisor,  good  performance  does  not  have  to  occur  at  the  minimum  of 
J(0).  The  supervisor  can  monitor  performance  during  each  step  of  an  iterative  tuning  procedure 
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and  stop  when  the  performance  is  good.  Since  the  adaptive  control  system  is  not  supposed  to  need 
a  supervisor,  the  cost  functional  for  adaptive  control  must  be  chosen  with  more  care  than  the  one 
for  supervised  automatic  tuning.  The  ability  to  construct  the  required  filters  and  the  function  f  in 
step  (3)  is  often  related  to  the  ability  to  do  off-line  automatic  tuning  because  the  cost  functional 
J(0)  often  has  the  form 


J(0)  =  avg[  J,(x(  .0))  ] 


(5.6) 


In  this  case,  we  take 


fT(w(k).0(k).x(k).<(k))  = 

0X 


(5.7) 


0-1 1 

In  Chapter  4.  for  example,  we  used  J,  =  .5(v(k)— ym(k))2  so  that  — =  y(k)— ym(k)  and  £  was 

0y 


used  to  approximate 


0y 


09- 


In  this  chapter  we  use  these  guidelines  to  design  a  slowly  adapting  control  system  for  a 
simplified  model  of  gasoline  engine  idle-speed  control. 


5.2.  Problem  Statement 


The  plant  and  controller  parametrization  are  given,  see  Fig.  5.1.  The  plant  uncertainty  is 
parametrized  by  the  vector  of  plant  parameters  p  =  [pj  p?  p3]T  with  nominal  value 
[0.67  0.017  0.75]T.  Each  element  of  p  can  vary  by  30%  of  its  nominal  value.  The  elements  Op 
02.  and  03  of  the  controller  parameter  vector  0  are  the  proportional  gain  from  the  output  y  to  the 
input  u2.  the  proportional  gain  from  y  to  the  input  u,.  and  the  integral  gain  from  v  to  the  input  u,. 
respectively.  A  state  space  representation  of  the  closed-loop  system  is 


x(k+l)  = 


/( k )  =  [ 


.994 
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Z 

* 

z— .994 

■5z  +  .2 _ 

z*  —  ,4z  +  .07 


a 

-1 

°3 

z  —  1 

Fig.  5.1.  Block  diagram  of  the  closed-loop  system  (5.8)  with  plant  parameters  p^  P2.  and  p3 
and  controller  parameters  9t.  92.  and  03. 

The  desired  value  of  the  output  is  zero.  However,  the  system  is  subject  to  infrequent  (separated  by 
more  than  50  samples)  step  changes  in  the  unmeasured  disturbance  d.  representing  load  changes. 
Good  tuning  of  the  controller  should  achieve  several  objectives  simultaneously.  The  response  of  y 
to  a  unit  step  change  in  d  at  time  k0  should  have  magnitude  less  than  0.1  for  all  k  ^  k„  +  25  and 
magnitude  less  than  0.01  for  all  k  ^  k,,  +  50.  The  response  should  be  well  damped.  The  closed- 
loop  eigenvalues  should  ali  have  magnitudes  less  than  0.9  so  that  integrator  windup  is  not  a 
problem.  These  three  objectives  were  stated  in  order  of  increasing  importance. 

In  terms  ol  the  guidelines  presented  in  the  introduction,  the  choice  of  the  controller 
parametrization  has  been  given  and  it  satisfies  the  smoothness  condition  (lb).  The  compatibility 
requirement  ( la)  that  the  controller  can  be  tuned  lor  different  values  of  the  plant  parameter  p  is  to 
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be  checked  by  actually  tuning  the  controller  for  different  values  of  p.  The  development  of  a  cost 
functional  with  the  properties  described  in  (2)  is  presented  as  an  iterative  procedure  in  which  an 
appropriate  cost  functional  is  determined  for  tuning  the  plant  with  the  nominal  value  of  p.  and 
then,  it  is  tested  to  verify  that  it  provides  good  tuning  for  all  possible  values  of  p.  The 
construction  of  a  parameter  update  law  is  straightforward  because  the  the  cost  functional  has  the 
form  (5.6). 

53.  Tuning  of  the  Nominal  Plant 

Using  the  method  of  sensitivity  points  (Kokotovic.  1973).  the  gradient  of  the  output  of  y(k) 
with  respect  to  constant  controller  parameters  is  vT(k)  =  [  Vj(k)  v2(k)  v3(k)  ],  where  v,.  v2.  and 
v3  are  signals  at  the  indicated  points  in  the  block  diagram  of  the  sensitivity  model.  Fig.  5.2.  A  state 


Fig.  5.2.  Block  diagram  ol  the  sensitivity  model  (5.9)  for  the  svstem  (5.S)  showing  the  sensi¬ 
tivity  points  V|,  v2.  vv 
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space  represemion  of  the  sensitivity  model  is 


.994  -p2(lO0,+.5p3)  -5p3  -.802  -502  -03 
1  0  0  0  0  0 

0  (1+Pi)P2  Pi  0  0  0 

^(k+l)  =  Q  0  0  0  .25  0  + 

0  .5p2  .5  -.28  .4  0 

0  0  0  .8  .5  1 

0  — 10p2  0  0  0  0 

v(k)  =  0  0  0  -.8  -.5  0 

0  0  0  0  0  -1 

Because  the  plant  parameter  vector  p  is  unknown,  this  sensitivity  model  cannot  be  realized  on  line. 
We  use  it  for  off-line  simulation  studies.  For  an  implementable  algorithm,  we  shall  use  (5.9)  with 
the  given  nominal  value  of  p  and  the  constant  value  of  0  which  gives  good  tuning  for  the  nominal 
plant. 

Since  the  input  d  is  not  measured,  the  usual  model  reference  approach  of  using  the  squared 
tracking  error  for  the  cost  functional  results  in 

J  =  avg[  y2()  ].  (5.10) 

For  testing  candidate  cost  functionals,  we  let  d(k)  be  a  square  wave  of  period  100  taking  values  1 
and  0.  The  average  of  y2  is  minimized  for  the  nominal  plant  by  the  controller  parameter  value 
0*  =  [  5.4  10.8  3.7  ]T.  As  shown  in  Fig.  5.3  the  response  is  oscillatory.  As  one  of  the  objectives  is 
to  have  a  well-damped  response,  something  needs  to  be  added  to  the  cost  functional  to  penalize  the 
oscillations. 

We  used  the  parameter  update  law 

0(k  +  l)  =  0(k)  — ev(k)y(k)  (5.11) 

and  slow  adaptation  to  search  for  0*.  Examining  the  response  for  different  values  of  0  along  the 
trajectory  of  the  slowly  adapting  system  (5.8).(5.9),(  s.l  1 ).  we  observe  that  for  some  values  ol  the 
parameters  the  response  is  close  to  that  of  a  well-damped  second  order  system  with  u.  o  zeros.  If 


Fig.  5.3.  The  response  of  the  y  over  one  period  of  d  for  9  -  [  5.4  10.8  3.7  ]T.  the  value  which 
minimizes  avg[  v2  ]. 

v(k)  was  indeed  the  output  of  a  second  order  system  with  two  zeros  and  input  d(k),  then  it  would 
satisfy 

y(k)  =  -a,y(k-l)-a2y(k-2)+b,d(k-l)+b2d(k-2)+b3d(k-3)  .  (5.12) 

Because  of  the  integral  feedback,  one  of  the  zeros  must  be  at  z  =  1  which  implies  that 
b|  +  b2+b3  =  0.  This,  in  turn,  implies  that  the  response  to  a  step  at  k  =  k,>  satisfies 

0  =  v(k)+aiv(k— 1  )+a2v(k— 2)  (5.13) 

for  all  k  -2  k . ,  +  3.  Hence,  using  the  equation  error 

e(k)  =  v(k)+a,y(k— l)+a2v(k— 2)  .  (5.13) 

we  can  incorporate  a  reterence  mode!  into  our  cost  functional.  With  some  experimentation,  we 
found  that  lor  the  nominal  plant  the  cost  functional 


Ill 

J  =  avg[  y2(  )  ]  +  a  avg[  e2(  )  ]  (5.14) 

with  the  relative  weighting  a  =  100  and  reference  model  coefficients  aj  =  —1.5  and  a 2  =  0.5725  has 
a  minimum  at  9*  =  [  3.0  3.5  1.34  ]T,  which  provides  the  good  response  shown  in  Fig. 5. 4.  The 
eigenvalues  of  the  closed-loop  system  (5.8)  all  have  magnitudes  less  than  0.80. 

5-4.  Tuning  of  All  Possible  Plants 

The  next  step  in  the  design  of  a  slowly  adapting  control  system  for  (5.1)  is  to  verify  that  the 
controller  can  be  tuned  for  all  possible  values  of  the  plant  parameters  and  to  check  that  the 
minimum  of  J  provides  a  good  controller  parameter  setting.  An  exhaustive  search  over  the  range  of 
plant  parameter  variations  reveals  that  the  closed-loop  system  (5.1)  with  controller  parameter 
fixed  at  9  =  6*  =  [  3.0  3.5  1.34  ]T  is  exponentially  stable  for  all  possible  plants.  This  is  important 
for  the  application  of  a  slowly  adapting  controller  because  it  suggests  that  we  may  be  able  to 
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Fig.  5.4.  The  response  of  v  over  one  period  of  d  for  9  =  [  3.0  3.5  1.34  ]T,  the  value  which 

minimizes  J  given  by  (5.14)  with  a  =  100.  at  =  —1.5.  ai  :=  0.5725. 
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initialize  our  controller  at  this  value  of  9  for  all  possible  plants.  Although  the  closed-loop  system 
has  all  its  eigenvalues  inside  the  unit  circle  for  all  possible  plants,  the  response  is  not  good  for  all 
possible  plants.  The  extreme  cases  are  generated  by  letting  the  plant  parameter  pT  take  the  values 
[0.871  0.0221  0.975  ],  [0.871  0.0119  0.525  ].  [0.469  0.0119  0.525  ].  and 

[  0.469  0.0221  0.975  ].  From  Fig.  5.5.  where  the  responses  for  the  nominal  value  of  p  and  each 
of  these  extreme  values  of  p  are  shown,  it  is  clear  that  the  plant  parameter  variations  are  significant 
enough  to  require  retuning  of  the  controller.  By  tuning  the  system,  we  verified  that  the  given 
controller  is  compatible  and  that  the  same  cost  functional  which  was  used  to  tune  the  nominal 
plant  can  be  used  to  tune  all  possible  plants.  The  tuned  responses,  which  are  shown  in  Fig.  5.6.  are 
very  good.  The  values  of  9  that  minimized  J  and  tuned  the  control  system  are  given  in  Table  5.1. 
For  each  tuned  system  the  eigenvalues  all  have  magnitudes  less  than  0.88. 

5  .5.  Simulation  Results  for  an  Implementable  Algorithm 

As  mentioned  before,  we  create  an  implementable  algorithm  by  using  the  nominal  value  of 


the  plant  parameter  vector  p  =  [  0.67  0.017  0.75  ]T  and  the  corresponding  value  of  the  controller 
parameter  vector  9  =  [  3.0  3.5  1.34  ]  in  the  sensitivity  model  (5.9).  In  order  to  differentiate  this 
approximate  gradient  from  the  true  gradient,  we  replace  v  by  £  as  the  output  of  (5.9).  Then,  the 
parameter  update  law  is  given  by 


Table  5.1.  The  nominal  plant  parameter  values  and  4  sets  of  plant  parameters  which  represent  ex¬ 
treme  changes  from  nominal  along  with  the  corresponding  value  of  the  controller 
parameter  after  tuning  to  minimize  the  cost  functional  J  in  (5.14)  with  a  =  100. 
a!  =  —1.5,  and  a2  =  0.5725. 
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0(k+l)  =  0(k)— er[£(k)y(k)  +  100[£(k)-1.5$(k-l  )+.5725£(k-2)]e(k)]  .  (5.15) 

By  experimentation  we  found  that  €  =  0.01  and  T  =  diag(  2  30  1.5  )  provided  good  parameter 
convergence  as  illustrated  by  the  trajectories  of  0  converging  from  its  tuned  value  for  the  nominal 
plant  to  its  tuned  value  for  each  of  the  extreme  plants  in  Figs.  5.7-5.10.  In  Fig.  5.7  the  controller 
parameters  0[  and  03  converge  quickly  with  monotonically  decreasing  average  values,  while  the 
parameter  02  converges  more  slowly  and  its  average  moves  initially  in  the  wrong  direction.  For 
this  value  p  =  [  .871  .0221  .975  ]T  the  output  is  not  very  sensitive  to  the  controller  parameter  02. 
This  can  be  seen  from  the  fact  that  the  output  changes  very  little  after  k=500.  but  02  does  not 
converge  until  after  k=1000.  In  Fig.  5.8  all  three  parameters  converge  very  quickly.  The  averages 
of  0!  and  03  are  again  monotonic,  while  the  average  of  02  overshoots  slightly  its  tuned  value  before 
converging  slowly  to  the  value  predicted  in  Table  5.1.  The  response  with  this  value 
p  =  [  871  .0119  .525  ]T  and  0  constant  at  its  nominal  value  0  =  [  3.0  3.5  1.34  ]T  is  the  large 
magnitude  oscillatory  response  in  Fig.  5.5.  Notice  that  with  adaptation  the  response  of  v  to  the 
change  in  d  at  k=50  is  almost  the  tuned  response.  This  is  the  plant  parameter  change  to  which  the 
controller  is  most  sensitive:  hence,  it  is  the  one  used  to  tune  the  gain  matrix  T  in  the  parameter 
update  law.  The  diagonal  elements  of  T  were  chosen  as  large  as  possible  without  causing  the 
parameters  to  significantly  overshoot  the  tuned  values.  The  convergence  of  the  parameters  shown 
in  Fig.  5.9  and  Fig.  5.10  is  about  the  same  speed  as  that  in  Fig.  5.7,  which  is  slower  than  that  in  Fig. 
5.8.  This  indicates  that  the  controller  is  less  sensitive  to  these  plant  parameter  changes. 

5.6.  Analysis  of  the  Implementable  Algorithm 

Suppose  that  p  is  given  and  fixed.  Let  0*  be  the  value  of  0  which  tunes  the  controller  for  the 
given  value  of  p.  We  denote  by  p11  and  0h  the  nominal  value  of  p  and  the  associated  tuned  value  ol 
0  By  design,  all  ot  the  assumptions  for  the  existence  ol  an  exponentially  attractive  integral 
manilold  are  satisfied  in  some  ball  around  0  in  the  parameter  space.  In  order  to  analyze  the 
behavior  in  the  mam'  ild.  we  applv  averaging  theorv.  We  could  analyze  the  slowly  adapting 
system  (5. 8). (5  91.(5.15  )  In  first  showing  that  £  is  a  good  approximation  of  the  true  gradient  v.  and 
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Fig.  5.9a.  The  controller  parameters  converging  from  nominal  values  to  the  tuned  values  for  the 
plant  parameter  vector  p  =  [  .469  .0119  .525]. 


Fig.  5.9b.  The  output  v  during  this  tuning  transient. 
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then,  using  J(0)  as  a  Lyapunov  function  to  quarantee  that  the  solutions  of  the  ODE  converge  to  a 
neighborhood  of  0  .  This  approach  relies  heavily  on  the  knowledge  of  J(0)  which  can  be  evaluated 
numerically  but  is  difficult  to  describe  analytically.  We  choose,  instead,  to  follow  the  analysis  in 


Chapter  4.  Letting 


Uk)  =  £(k)  -  1.5£(k— 1)  +  .5725{(k— 2) 


(5.16) 


the  averaged  system  is  described  by 


f(9)  =  — avg[  £()y(-)  ]  -  aavg[  £e(  )e(  )  ]  . 


Defining 


(5.17) 


£(k.p.0.pn.0n)  = 


_  ,5z4-.2  ,5(z+l) 

z2— .4z+.07  z— p,n 

.5z+.2  5(z+l)  1 

z2—  4z+.07  z— p"  z— 1 


~  WCL(  pn  .0”  .z  )y(  k .  p  .0  ) 


(5.18) 


where  WCL  is  the  transfer  function  from  d  to  y. 


WCL(p.0.z)  = 


z— .994 


1+J-! — *  100,-t-  p  +  +  — - : 1 

z  z— .994  z-p,  F  *  z-1  z2- 


,5z4-.2 


z~— .4z4-.07 


(5.19) 


we  rewrite  (5.17)  in  the  form 


f(0)  =  — R(p.0.pn.0",0*)( 0—0*)  -  b(p,0,pn,0n.0*) 


(5.20) 


R(p.0.pn.0'',0*)  =  avg[  t,(  .p.0,pn.0")£T(  ,p.0,p,0*)  ] 

+  aavg[  {„(  .p.0.p".0")(;tT(\p.0.p.0*) 


(5.21) 


b(  p.0.p,‘.0,1.0*)  =  avg[  £(-,p,0.pn,0n)y(-.p.0*)  ] 

+  aavg[  (;,,(•. p.0.pn.0,1)e(-.p.0*)  ]  . 


(5.22) 


The  analysis  then  proceeds  as  in  Chapter  4  with  Theorem  4.2  providing  a  sufficient  condition  for 


I*?*  TV.S 
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by: 


the  exponential  stability  of  an  invariant  set  containing  0*.  From  the  closeness  of  the  tuned 
responses  in  Fig.  5.6.  we  conclude  that  WCL(pn.0n.z)y(k.p.0)  =  WCL(p.0*,z)y(k.p,0)  for  all  0  in  a 
ball  around  0  .  Then,  the  difference  between  £(k,p,0.pn.0n)  and  £(k.p.0.p.0*)  is  due  to  the  difference 


in  the  transfer  functions  — - —  and  — ? — .  Since  £»  is  simply  a  moving  average  of  £.  the  difference 

z-Pln  z-p. 


between  £e(k.p,0,pn.0n)  and  £e(k,p.0.p,0  )  is  also  due  to  the  difference  in  these  two  transfer 
functions.  The  fact  that  pi  varies  no  more  than  30%  from  p"  ensures  that  the  matrix  R  +  RT  is 
positive  semidefinite.  From  the  convergence  of  the  parameters  in  Figs.  5.7-5.10.  we  conclude  that  £ 
is  persistently  exciting  for  the  three  controller  parameters.  This  implies  that  R  +  RT  is  in  fact 
positive  definite.  Hence  R  satifies  the  hypotheses  of  Theorem  4.2  and  our  analysis  agrees  with  our 
simulations.  We  remark  that  with  the  computer-aided  design  tools  available  today  it  is  more 
efficient  to  estimate  via  simulation  and  other  numerical  tests  the  size  of  the  balls  around  0*  which 
arise  in  the  analysis  than  to  estimate  these  balls  analytically. 


5.7.  Concluding  Remarks 

In  this  chapter  we  designed  a  slowly  adapting  control  system  for  a  given  plant  with  uncertain 
parameters  and  a  given  controller  parametrization.  We  illustrated  the  use  of  an  equation  error 
approach  for  including  reference  model  information  in  the  cost  functional  used  for  tuning  the 
system.  Using  our  guidelines  for  the  development  of  a  slowly  adapting  system,  we  were  able  to 
make  use  of  a  priori  information  in  the  design,  analysis,  and  testing  phases  of  the  development. 
We  point  out  that  many  of  the  steps  involved  in  developing  a  slowly  adapting  control  system  are 
already  included  in  the  design  of  fixed  parameter  control  systems.  Finally,  we  emphasize  that  by 
taking  advantage  of  slow  adaptation,  we  can  develop  adaptive  control  systems  for  controllers  with 
given  structures.  Hence,  existing  fixed  gain  control  systems  can  be  upgraded  to  slowly  adapting 
control  systems  without  reparametrization. 
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